Welcome to TinyMS’s documentation!¶
TinyMS is an Easy-to-Use deep learning development toolkit based on MindSpore, designed to providing quick-start guidelines for machine learning beginners.
What is TinyMS¶
TinyMS Project¶
Introduction¶
TinyMS is an open source deep learning development kit written in Python. It runs on top of deep learning framework like MindSpore, and provides high level API covers the entire lifecycle and workflow for AI development that ranges from data preparation to model deployment.
TinyMS is composed of several modules including data, model and serving. It provides transform data processing operators for different scenarios and reuses MindSpore datasets like cifar-10.
TinyMS are mainly designed to help user groups like deep learning beginners, researchers who conducts studies related to deep learning, and AI application developers with a crash course.
Combined with video tutorials (available in Chinese on Bilibili site and soon with English version on YouTube channel), TinyMS project offers the best deep learning crash course and entry level development experience todate.
TinyMS vs Keras¶
To quote from Keras’ website
Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result as fast as possible is key to doing good research.
Keras is known for its completeness. Keras is composed of modules such as dataset, layer, model and backend. It provides commonly used datasets and data processing functions for different scenarios. The layer module provides all encompassing functionalities such as convolution, embedding, pooling, backend (multiple support for Tensorflow, CNTK and Theano). The model module provides functionalities such as model types (sequential or functional), network construction (input, output, pooling, etc.), model compilation, model training, model verification and inference.
In comparison, TinyMS designs a set of more abstract high level APIs and therefore is less complex than Keras. For example one can complete dataset preprocessing with just one line of code in TinyMS. TinyMS also provides several individual tools and quick model deployment module which Keras has not yet offered.
TinyMS vs Fastai¶
To quote from fastai’s README
fastai is a deep learning library which provides practitioners with high-level components that can quickly and easily provide state-of-the-art results in standard deep learning domains, and provides researchers with low-level components that can be mixed and matched to build new approaches.
With the strength of PyTorch’s flexibility, Fastai could provide out-of-the-box development experience for model types like vision, text, tabular, collab. However unlike Keras’s multi-backend support, Fastai’s backend is tightly coupled with PyTorch and its versioning.
Fastai is known for its “petitness” which provides a great lightweight and easy-to-understand structure. Fastai consists of three major modules: data, models and learner. The data module provide transform data preprocessing operations which is convenient for developers. The model module provides many predefined networks like unet for quick model construction. The learner module defines the relationship between data and model with a set of callback functions to help developers quickly grasp the most common deep learning architectures. It provides a rich set of tools such as data downloading, decompression, figure verification and file processing.
In comparison, while sharing similar design concepts on high level APIs, TinyMS offers predefined MindSpore datasets which could help developers with dataset processing enormously as well as quick model deployment module, both of which Fastai has not yet provided.
TinyMS Community¶
Other than TinyMS project，the community at-large also includes the many other projects and activities：
Specification project：an attempt to standardize the format of model training scripts. Due to the abstract nature of TinyMS APIs, we found it necessary to have a standard or guideline for ModelZoo.
https://tinyms-ai.github.io：a simple website built in open source based upon GitHub Page mechanism.
RustedAI Team：only visible for org members at the moment, RustedAI is an initiative that TinyMS tries to build for more adoption of Rust-lang in the field of deep learning to meet the goal of low runtime footprint.
Community Activities：We will organize TinyMS model competitions and many other activities including Meetups, webinars, etc.
TinyMS and Developers¶
As a new open source project, TinyMS stands on the shoulder of the giants like Keras and Fastai. Although we hope to achieve many innovations in our design, it still depends on a vibrant community and ecosystem to make TinyMS reach the depth and broadness of its predecessors in order to better serve the academia, industry and developers in general.
Install TinyMS¶
Installation For Beginners¶
Pypi¶
For users who own a clean environment, it is recommended to use pypi to install TinyMS given that the following requirements are meet. For those who don’t, Anaconda is a good choice for setting up the python environment.
Prerequisites
OS:
Ubuntu 18.04orWindows 10Python:
3.7.5
For China based users it is recommended to run the following command lines to help with faster download
mkdir -pv /root/.pip \
&& echo "[global]" > /root/.pip/pip.conf \
&& echo "trusted-host=mirrors.aliyun.com" >> /root/.pip/pip.conf \
&& echo "index-url=http://mirrors.aliyun.com/pypi/simple/" >> /root/.pip/pip.conf
pip install tinyms==0.1.0
Docker¶
For those who don’t want to affect the local develop environment due to difficulty of meeting the prerequisites, using docker to install is recommended
docker:
v18.06.1-ce
If user wants to try the tutorials that are written in .ipynb files，please pull jupyter version of TinyMS in which jupyter components are installed by default
Default version
docker pull tinyms/tinyms:0.1.0
Jupyter version
If user wants to try jupyter, run the following command line
docker pull tinyms/tinyms:0.1.0-jupyter
docker run -it --net=host tinyms/tinyms:0.1.0-jupyter
Open a browser on the local machine, type in
<Your_external_IP_address>:8888
Example: 188.8.8.88:8888, the default password is tinyms，then user can log in to jupyter
Installation For Experienced Developers¶
For developers who want to develop based on TinyMS, install from source
sudo apt-get install -y libssl-dev
git clone https://github.com/tinyms-ai/tinyms.git
cd tinyms
pip install -r requirements.txt
python setup.py install
Validate installation¶
Create a python or jupyter kernel, input the following codes
import tinyms as ts
from tinyms.primitives import tensor_add
x = ts.ones([2, 3])
y = ts.ones([2, 3])
print(tensor_add(x, y))
If the output is similar to below, then the installation is valid
[[2. 2. 2.]
[2. 2. 2.]]
Implementing an Image Classification App in One Minute¶
In this tutorial, constructing a LeNet5 model, downloading dataset, training, starting the server and making predictions of the model using TinyMS API will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of six parts, constructing the model, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import json
import tinyms.optimizers as opt
from PIL import Image
from tinyms import context
from tinyms.data import MnistDataset, download_dataset
from tinyms.vision import mnist_transform, ImageViewer
from tinyms.model import Model, lenet5
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.metrics import Accuracy
from tinyms.losses import SoftmaxCrossEntropyWithLogits
from tinyms.callbacks import ModelCheckpoint, CheckpointConfig, LossMonitor
[WARNING] ME(14780:139834376955712,MainProcess):2021-03-19-15:56:12.182.640 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Construct the model¶
TinyMS encapsulates init and construct of the LeNet5 model, the line of the code is reduced to construct the LeNet5 model:
[2]:
# build the network
net = lenet5(class_num=10)
model = Model(net)
2. Download dataset¶
The MNIST dataset will be downloaded if mnist folder didn’t exist at the root. If mnist folder already exists, this step will not be performed.
[3]:
# download the dataset
mnist_path = '/root/mnist'
if not os.path.exists(mnist_path):
download_dataset('mnist', '/root')
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
************Dataset already exists.**************
3. Train the model & evaluation¶
The dataset for both training and evaluation will be defined here, and the parameters for training also set in this block. A trained ckpt file will be saved to /etc/tinyms/serving/lenet5 folder for later use, meanwhile the evaluation will be performed and the Accuracy can be checked
[ ]:
# check lenet folder exists or not
ckpt_folder = '/etc/tinyms/serving/lenet5'
ckpt_path = '/etc/tinyms/serving/lenet5/lenet5.ckpt'
if not os.path.exists(ckpt_folder):
!mkdir -p /etc/tinyms/serving/lenet5
else:
print('lenet5 ckpt folder already exists')
# set environment parameters
device_target = "CPU"
context.set_context(mode=context.GRAPH_MODE, device_target=device_target)
dataset_sink_mode = False
# define the training and evaluation dataset
train_dataset = MnistDataset(os.path.join(mnist_path, "train"), shuffle=True)
train_dataset = mnist_transform.apply_ds(train_dataset)
eval_dataset = MnistDataset(os.path.join(mnist_path, "test"), shuffle=True)
eval_dataset = mnist_transform.apply_ds(eval_dataset)
# parameters for training
lr = 0.01
momentum = 0.9
epoch_size = 1
batch_size = 32
# define the loss function
net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
# define the optimizer
net_opt = opt.Momentum(net.trainable_params(), lr, momentum)
net_metrics={"Accuracy": Accuracy()}
model.compile(loss_fn=net_loss, optimizer=net_opt, metrics=net_metrics)
print('************************Start training*************************')
ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", config=CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10))
model.train(epoch_size, train_dataset, callbacks=[ckpoint_cb, LossMonitor()],dataset_sink_mode=dataset_sink_mode)
print('************************Finished training*************************')
model.save_checkpoint(ckpt_path)
model.load_checkpoint(ckpt_path)
print('************************Start evaluation*************************')
acc = model.eval(eval_dataset, dataset_sink_mode=dataset_sink_mode)
print("============== Accuracy:{} ==============".format(acc))
4. Define servable.json¶
Define the lenet5 servable json file for model name, format and number of classes for serving.
[5]:
servable_json = [{'name': 'lenet5',
'description': 'This servable hosts a lenet5 model predicting numbers',
'model': {
"name": "lenet5",
"format": "ckpt",
"class_num": 10}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
5. Start server¶
5.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which constructs the model, run the prediction and send the result back to the client.
5.2 Start server¶
Run the following code block to start the server:
[6]:
start_server()
Server starts at host 127.0.0.1, port 5000
6. Make predictions¶
6.1 Upload the pic¶
A picture of a single digit number is required to be the input. The picture we use in this tutorial can be found HERE, then save the picture to the root folder, and rename it to 7.png (or any other name you like).
Or run the following code to download the pic for this tutorial:
[7]:
if not os.path.exists('/root/7.png'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/numbers/7.png
else:
print('7.png already exists')
--2021-03-19 15:56:37-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/numbers/7.png
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.90, 49.4.112.113, 121.36.121.44, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.90|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 34970 (34K) [image/png]
Saving to: ‘/root/7.png’
7.png 100%[===================>] 34.15K --.-KB/s in 0.1s
2021-03-19 15:56:37 (345 KB/s) - ‘/root/7.png’ saved [34970/34970]
6.2 List servables¶
Use list_servables function to check what model is being served right now.
[8]:
list_servables()
[8]:
[{'description': 'This servable hosts a lenet5 model predicting numbers',
'model': {'class_num': 10, 'format': 'ckpt', 'name': 'lenet5'},
'name': 'lenet5'}]
If the output description shows it is a lenet5 model, then we can continue to next step to send our request.
6.3 Sending request and get the result¶
Run predict function to send the request, select between TOP1_CLASS and TOP5_CLASS:
[9]:
image_path = "/root/7.png"
strategy = "TOP1_CLASS"
# predict(image_path, servable_name, dataset='mnist', strategy='TOP1_CLASS')
if server_started() is True:
img_viewer = ImageViewer(Image.open(image_path), image_path)
img_viewer.show()
print(predict(image_path, 'lenet5', 'mnist', strategy))
else:
print("Server not started")
TOP1: 7, score: 0.99977773427963256836
If user can see the output similar to this:
TOP1: 7, score: 0.99934917688369750977
that means the prediction is successfully performed
TinyMS ResNet50 Tutorial¶
In this tutorial, using TinyMS API to train/serve a ResNet50 model will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of six parts, constructing the model, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import json
from PIL import Image
from tinyms import context
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.data import Cifar10Dataset, download_dataset, ImageFolderDataset
from tinyms.vision import cifar10_transform, ImageViewer, imagefolder_transform
from tinyms.model import Model, resnet50
from tinyms.callbacks import ModelCheckpoint, CheckpointConfig, LossMonitor
from tinyms.metrics import Accuracy
from tinyms.optimizers import Momentum
from tinyms.losses import SoftmaxCrossEntropyWithLogits
[WARNING] ME(12569:140321685239616,MainProcess):2021-03-19-15:21:33.633.399 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Construct the model¶
TinyMS encapsulates init and construct of the ResNet50 model, the line of the code is reduced to construct the model:
[2]:
# build the network
net = resnet50(class_num=10)
model = Model(net)
2. Download dataset¶
Training ResNet50 model with cifar10 dataset will be demonstrated, while we provide two pre-trained ckpt files for users to download, one is trained with cifar10 dataset and the other one is trained with ImageNet2012 dataset.
[3]:
# download the cifar10 dataset
cifar10_path = '/root/cifar10/cifar-10-batches-bin'
if not os.path.exists(cifar10_path):
download_dataset('cifar10', '/root')
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
************** Downloading the Cifar10 dataset **************
[███████████████████████████████████████████████████████████████████████████████████████████████████ ] 99.81%************Download complete*************
3. Train the model & evaluation¶
The dataset for both training and evaluation will be defined here, and the parameters for training also set in this block. A trained ckpt file will be saved to /etc/tinyms/serving/resnet50_cifar10 folder for later use, meanwhile the evaluation will be performed and the Accuracy can be checked
Notice: Since training ResNet50 on CPU is time consuming, we recommend skip training and using provided ckpt files to run.
[ ]:
# check ckpt folder exists or not
cifar10_ckpt_folder = '/etc/tinyms/serving/resnet50_cifar10'
cifar10_ckpt_path = '/etc/tinyms/serving/resnet50_cifar10/resnet50.ckpt'
if not os.path.exists(cifar10_ckpt_folder):
!mkdir -p /etc/tinyms/serving/resnet50_cifar10
else:
print('resnet50_cifar10 ckpt folder already exists')
epoch_size = 90 # default is 90
batch_size = 32
# set environment parameters
dataset_sink_mode = False
device_target = "CPU"
context.set_context(mode=context.GRAPH_MODE, device_target=device_target)
# set dataset parameters
train_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
train_dataset = cifar10_transform.apply_ds(train_dataset, repeat_size=1, batch_size=32, is_training=True)
eval_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
eval_dataset = cifar10_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=32, is_training=False)
step_size = train_dataset.get_dataset_size()
save_checkpoint_epochs = 5
ckpoint_cb = ModelCheckpoint(prefix="resnet_cifar10", config=CheckpointConfig(
save_checkpoint_steps=save_checkpoint_epochs * train_dataset.get_dataset_size(),
keep_checkpoint_max=10))
# define the loss function
net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
# define the optimizer
net_opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), 0.01, 0.9)
model.compile(loss_fn=net_loss, optimizer=net_opt, metrics={"Accuracy": Accuracy()})
print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=[ckpoint_cb, LossMonitor()],dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(cifar10_ckpt_path)
print('************************Finished training*************************')
model.load_checkpoint(cifar10_ckpt_path)
print('************************Start evaluation*************************')
acc = model.eval(eval_dataset, dataset_sink_mode=dataset_sink_mode)
print("============== Accuracy:{} ==============".format(acc))
Notice: If skipped training process, download the pretrained ckpt files and continue to serving
Click resnet_imagenet to download resnet-imagenet ckpt file and click resnet_cifar to download resnet-cifar ckpt file. Save this file to /etc/tinyms/serving/resnet50_<dataset_name>/resnet50.ckpt.
Or run the following code to download the resnet_imagenet and resnet_cifar ckpt file:
[4]:
# check lenet folder exists or not, and download resnet50_imagenet
imagenet2012_ckpt_folder = '/etc/tinyms/serving/resnet50_imagenet2012'
imagenet2012_ckpt_path = '/etc/tinyms/serving/resnet50_imagenet2012/resnet50.ckpt'
if not os.path.exists(imagenet2012_ckpt_folder):
!mkdir -p /etc/tinyms/serving/resnet50_imagenet2012
!wget -P /etc/tinyms/serving/resnet50_imagenet2012 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/imagenet2012/resnet50.ckpt
else:
print('imagenet2012 ckpt folder already exists')
if not os.path.exists(imagenet2012_ckpt_path):
!wget -P /etc/tinyms/serving/resnet50_imagenet2012 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/imagenet2012/resnet50.ckpt
else:
print('imagenet2012 ckpt file already exists')
# check lenet folder exists or not
cifar10_ckpt_folder = '/etc/tinyms/serving/resnet50_cifar10'
cifar10_ckpt_path = '/etc/tinyms/serving/resnet50_cifar10/resnet50.ckpt'
if not os.path.exists(cifar10_ckpt_folder):
!mkdir -p /etc/tinyms/serving/resnet50_cifar10
!wget -P /etc/tinyms/serving/resnet50_cifar10 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/resnet50.ckpt
else:
print('cifar10 ckpt folder already exists')
if not os.path.exists(cifar10_ckpt_path):
!wget -P /etc/tinyms/serving/resnet50_cifar10 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/resnet50.ckpt
else:
print('cifar10 ckpt file already exists')
imagenet2012 ckpt folder already exists
--2021-03-19 15:23:45-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/imagenet2012/resnet50.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 188521005 (180M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/resnet50_imagenet2012/resnet50.ckpt’
resnet50.ckpt 100%[===================>] 179.79M 36.7MB/s in 5.9s
2021-03-19 15:23:52 (30.4 MB/s) - ‘/etc/tinyms/serving/resnet50_imagenet2012/resnet50.ckpt’ saved [188521005/188521005]
cifar10 ckpt folder already exists
--2021-03-19 15:23:52-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/resnet50.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 188462121 (180M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/resnet50_cifar10/resnet50.ckpt’
resnet50.ckpt 100%[===================>] 179.73M 35.7MB/s in 5.6s
2021-03-19 15:23:58 (32.3 MB/s) - ‘/etc/tinyms/serving/resnet50_cifar10/resnet50.ckpt’ saved [188462121/188462121]
4. Define servable.json¶
Choose only one of the following two code blocks to define the servable json file which will be used later
Run this code to define the servable json file for ResNet50_imagenet2012 model:
[5]:
servable_json = [{'name': 'resnet50_imagenet2012',
'description': 'This servable hosts a resnet50 model predicting mushrooms',
'model': {
"name": "resnet50",
"format": "ckpt",
"class_num": 9}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
Or run this code to define the servable json file for ResNet50_cifar10 model:
[ ]:
servable_json = [{'name': 'resnet50_cifar10',
'description': 'This servable hosts a resnet50 model predicting 10 classes of objects',
'model': {
"name": "resnet50",
"format": "ckpt",
"class_num": 10}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
5. Start server¶
5.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. There is a server and client. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which will construct the model, run the prediction and send the result back to the client.
5.2 start server¶
Run the following code block to start the server:
[6]:
start_server()
Server starts at host 127.0.0.1, port 5000
6. Make predictions¶
6.1 Upload the pic¶
A picture is required to be the input. The resnet_imagenet ckpt requires a mushroom picture, while the resnet_cifar ckpt requires a picture of
['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Click mushroom which is used in this tutorial for resnet_imagenet and airplane for resnet-cifar. Upload the pic, if using terminal, either scp or wget will do, if running in Jupyter, click Upload button at the top right and select the picture. Save the picture at the root
folder, rename to mushroom.jpeg(for resnet-imagenet) or airplane.jpg(for resnet-cifar).
Or run this code to download mushroom pic for resnet_imagenet and airplane for resnet_cifar:
[7]:
# download mushroom pic
if not os.path.exists('/root/mushroom.jpeg'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/mushrooms/mushroom.jpeg
else:
print('mushroom.jpeg already exists')
# download airplane pic
if not os.path.exists('/root/airplane.jpg'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
else:
print('airplane.jpg already exists')
--2021-03-19 15:24:11-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/mushrooms/mushroom.jpeg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 76020 (74K) [image/jpeg]
Saving to: ‘/root/mushroom.jpeg’
mushroom.jpeg 100%[===================>] 74.24K 370KB/s in 0.2s
2021-03-19 15:24:12 (370 KB/s) - ‘/root/mushroom.jpeg’ saved [76020/76020]
--2021-03-19 15:24:12-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 151188 (148K) [image/jpeg]
Saving to: ‘/root/airplane.jpg’
airplane.jpg 100%[===================>] 147.64K 561KB/s in 0.3s
2021-03-19 15:24:12 (561 KB/s) - ‘/root/airplane.jpg’ saved [151188/151188]
6.2 List servables¶
Now, use list_servables function to check what model is servable right now.
[8]:
list_servables()
[8]:
[{'description': 'This servable hosts a resnet50 model predicting mushrooms',
'model': {'class_num': 9, 'format': 'ckpt', 'name': 'resnet50'},
'name': 'resnet50_imagenet2012'}]
If the output description shows it is a resnet50 model, run the following code which will automatically detect whether it is a imagenet model or a cifar model
6.3 Sending request and get the result¶
Run predict function to send the request, select between TOP1_CLASS and TOP5_CLASS to check the output
[9]:
# set image_path and output strategy
imagenet_image_path = "/root/mushroom.jpeg"
cifar_image_path = "/root/airplane.jpg"
strategy = "TOP1_CLASS"
# predict(image_path, servable_name, dataset_name, strategy='TOP1_CLASS')
if server_started() is True:
servable_name = list_servables()[0]['name']
if servable_name == 'resnet50_imagenet2012':
img_viewer = ImageViewer(Image.open(imagenet_image_path), imagenet_image_path)
img_viewer.show()
print(predict(imagenet_image_path, "resnet50_imagenet2012", "imagenet2012", strategy))
else:
img_viewer = ImageViewer(Image.open(cifar_image_path), cifar_image_path)
img_viewer.show()
print(predict(cifar_image_path, "resnet50_cifar10", 'cifar10', strategy))
else:
print('Server not started')
TOP1: Amanita毒蝇伞,伞菌目,鹅膏菌科,鹅膏菌属,主要分布于我国黑龙江、吉林、四川、西藏、云南等地,有毒, score: 0.99750286340713500977
Check output¶
If user runs resnet_imagenet and see output similar to this:
TOP1: Amanita毒蝇伞,伞菌目,鹅膏菌科,鹅膏菌属,主要分布于我国黑龙江、吉林、四川、西藏、云南等地,有毒, score: 0.99119007587432861328
that means the prediction result is returned and the inference is completed.
Or user runs resnet_cifar, and the output is expected to be like this:
TOP1: airplane, score: 0.99997282028198242188
## Change model
Run the following code to train a ResNet50 model with ImageNet2012 mushroom dataset, then run another servable_json code block to define servable. The dataset will be downloaded here.
[ ]:
# download the imagenet2012 mushroom dataset
imagenet_path = '/root/mushrooms'
if not os.path.exists(imagenet_path):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/resnet-50/mushrooms/mushrooms.zip
!mkdir /root/mushrooms/
!unzip /root/mushrooms.zip -d /root/mushrooms/
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
# check ckpt folder exists or not
imagenet_ckpt_folder = '/etc/tinyms/serving/resnet50_imagenet2012'
imagenet_ckpt_path = '/etc/tinyms/serving/resnet50_imagenet2012/resnet50.ckpt'
if not os.path.exists(imagenet_ckpt_folder):
!mkdir -p /etc/tinyms/serving/resnet50_imagenet2012
else:
print('resnet50_imagenet2012 ckpt folder already exists')
epoch_size = 90 # default is 90
batch_size = 32
# set environment parameters
dataset_sink_mode = False
device_target = "CPU"
context.set_context(mode=context.GRAPH_MODE, device_target=device_target)
# set dataset parameters
imagenet_train_path = '/root/mushrooms/train'
train_dataset = ImageFolderDataset(imagenet_train_path, num_parallel_workers=4, shuffle=True)
train_dataset = imagefolder_transform.apply_ds(train_dataset, repeat_size=1, batch_size=32, is_training=True)
imagenet_eval_path = '/root/mushrooms/eval'
eval_dataset = ImageFolderDataset(imagenet_eval_path, num_parallel_workers=4, shuffle=True)
eval_dataset = imagefolder_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=32, is_training=False)
step_size = train_dataset.get_dataset_size()
save_checkpoint_epochs = 5
ckpoint_cb = ModelCheckpoint(prefix="resnet_imagenet2012", config=CheckpointConfig(
save_checkpoint_steps=save_checkpoint_epochs * train_dataset.get_dataset_size(),
keep_checkpoint_max=10))
# define the loss function
net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean")
# define the optimizer
net_opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), 0.01, 0.9)
model.compile(loss_fn=net_loss, optimizer=net_opt, metrics={"Accuracy": Accuracy()})
print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=[ckpoint_cb, LossMonitor()],dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(imagenet_ckpt_path)
print('************************Finished training*************************')
model.load_checkpoint(imagenet_ckpt_path)
print('************************Start evaluation*************************')
acc = model.eval(eval_dataset, dataset_sink_mode=dataset_sink_mode)
print("============== Accuracy:{} ==============".format(acc))
TinyMS MobileNetV2 Tutorial¶
In this tutorial, using TinyMS API to train/serve a MobileNetV2 model will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of six parts, constructing the model, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import json
from PIL import Image
from tinyms import context
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.model import Model, mobilenetv2
from tinyms.data import Cifar10Dataset, download_dataset
from tinyms.vision import cifar10_transform, ImageViewer
from tinyms.metrics import Accuracy
from tinyms.optimizers import Momentum
from tinyms.losses import CrossEntropyWithLabelSmooth
from tinyms.utils.train.loss_manager import FixedLossScaleManager
from tinyms.utils.train.lr_generator import mobilenetv2_lr
from tinyms.utils.train.cb_config import mobilenetv2_cb
[WARNING] ME(8895:140407127349056,MainProcess):2021-03-19-15:03:40.515.970 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Construct the model¶
TinyMS encapsulates init and construct of the MobileNetV2 model, the line of the code is reduced to construct the model:
[2]:
# build the model
net = mobilenetv2(class_num=10, is_training=True)
model = Model(net)
2. Download dataset¶
The cifar10 dataset will be downloaded if cifar10 folder didn’t exist at the root. If cifar10 folder already exists, this step will not be performed.
[3]:
# download the dataset
cifar10_path = '/root/cifar10/cifar-10-batches-bin'
if not os.path.exists(cifar10_path):
download_dataset('cifar10', '/root')
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
************** Downloading the Cifar10 dataset **************
[████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
============== /root/cifar10/cifar-10-binary.tar.gz is ready ==============
************Download complete*************
3. Train the model & evaluation¶
The dataset for both training and evaluation will be defined here, and the parameters for training also set in this block. A trained ckpt file will be saved to /etc/tinyms/serving/mobilenetv2 folder for later use, meanwhile the evaluation will be performed and the Accuracy can be checked
Notice: Since training MobileNetV2 on CPU is time consuming, we recommend skip training and using provided ckpt files to run.
[ ]:
# check ckpt folder exists or not
ckpt_folder = '/etc/tinyms/serving/mobilenetv2'
ckpt_path = '/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt'
if not os.path.exists(ckpt_folder):
!mkdir -p /etc/tinyms/serving/mobilenetv2
else:
print('mobilenetv2 ckpt folder already exists')
# Declare common variables
epoch_size = 60 # default is 60
batch_size = 32
class_num = 10
# set runtime environment
device_target="CPU"
dataset_sink_mode = False
context.set_context(mode=context.GRAPH_MODE, device_target=device_target)
# create cifar10 dataset
train_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
train_dataset = cifar10_transform.apply_ds(train_dataset, repeat_size=1, batch_size=32, is_training=True)
eval_dataset = Cifar10Dataset(cifar10_path, num_parallel_workers=4, shuffle=True)
eval_dataset = cifar10_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=32, is_training=False)
step_size = train_dataset.get_dataset_size()
# define the loss function
label_smooth = 0.1
loss = CrossEntropyWithLabelSmooth(smooth_factor=label_smooth,num_classes=class_num)
# get learning rate
lr_max = 0.001
lr_init_scale = 0.01
lr_end_scale = 0.01
lr = mobilenetv2_lr(global_step=0,
lr_init=lr_max*lr_init_scale,
lr_end=lr_max*lr_end_scale,
lr_max=lr_max,
warmup_epochs=2,
total_epochs=epoch_size,
steps_per_epoch=step_size)
# define the optimizer
loss_scale = FixedLossScaleManager(1024, drop_overflow_update=False)
opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()),lr, 0.9, 4e-5, 1024)
model.compile(loss_fn=loss, optimizer=opt, metrics={"Accuracy": Accuracy()},loss_scale_manager=loss_scale)
# configure checkpoint to save weights and do training job
save_checkpoint_epochs = 10
ckpoint_cb = mobilenetv2_cb(device_target=device_target,
lr=lr,
is_saving_checkpoint=True,
save_checkpoint_epochs=save_checkpoint_epochs,
step_size=step_size)
print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=ckpoint_cb, dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(ckpt_path)
print('************************Finished training*************************')
model.load_checkpoint(ckpt_path)
print('************************Start evaluation*************************')
acc = model.eval(eval_dataset, dataset_sink_mode=dataset_sink_mode)
print("============== Accuracy:{} ==============".format(acc))
Notice: If skipped training process, download the pretrained ckpt file and continue to serving
Click HERE to download MobileNetV2 ckpt file and save this file to /etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt.
Or run the following code to download and store the ckpt file:
[4]:
mobilenetv2_ckpt_folder = '/etc/tinyms/serving/mobilenetv2'
mobilenetv2_ckpt_path = '/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt'
if not os.path.exists(mobilenetv2_ckpt_folder):
!mkdir -p /etc/tinyms/serving/mobilenetv2
!wget -P /etc/tinyms/serving/mobilenetv2 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/mobilenetv2.ckpt
else:
print('mobilenetv2 ckpt folder already exists')
if not os.path.exists(mobilenetv2_ckpt_path):
!wget -P /etc/tinyms/serving/mobilenetv2 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/mobilenetv2.ckpt
else:
print('mobilenetv2 ckpt file already exists')
mobilenetv2 ckpt folder already exists
--2021-03-19 15:06:26-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cifar10/mobilenetv2.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.5, 49.4.112.90, 49.4.112.113, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.5|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 18509001 (18M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt’
mobilenetv2.ckpt 100%[===================>] 17.65M 16.1MB/s in 1.1s
2021-03-19 15:06:29 (16.1 MB/s) - ‘/etc/tinyms/serving/mobilenetv2/mobilenetv2.ckpt’ saved [18509001/18509001]
4. Define servable.json¶
Define the MobileNetV2 servable json file for model name, format and number of classes for later use.
[5]:
servable_json = [{'name': 'mobilenetv2',
'description': 'This servable hosts a mobilenetv2 model predicting 10 classes of objects',
'model': {
"name": "mobilenetv2",
"format": "ckpt",
"class_num": 10}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
5. Start server¶
5.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. There is a server and client. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which will construct the model, run the prediction and send the result back to the client.
5.2 start server¶
Run the following code block to start the server:
[6]:
start_server()
Server starts at host 127.0.0.1, port 5000
6. Make predictions¶
6.1 Upload the pic¶
A picture is required to be the input. This ckpt requires a picture containing the following objects:
['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Click airplane to download the picture used in this tutorial. Upload the pic, if using terminal, either scp or wget will do, if running in Jupyter, click Upload button at the top right and select the picture.
Save the picture at the root folder, rename to airplane.jpg(or any name you want).
Or run the following code to download the picture used in this tutorial
[7]:
# download the airplane pic
if not os.path.exists('/root/airplane.jpg'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
else:
print('airplane.jpg already exists')
--2021-03-19 15:06:37-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/objects/airplane.jpg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.5, 49.4.112.90, 49.4.112.113, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.5|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 151188 (148K) [image/jpeg]
Saving to: ‘/root/airplane.jpg’
airplane.jpg 100%[===================>] 147.64K 560KB/s in 0.3s
2021-03-19 15:06:38 (560 KB/s) - ‘/root/airplane.jpg’ saved [151188/151188]
6.2 List servables¶
Now, we can use list_servables function to check what model is servable right now.
[8]:
list_servables()
[8]:
[{'description': 'This servable hosts a mobilenetv2 model predicting 10 classes of objects',
'model': {'class_num': 10, 'format': 'ckpt', 'name': 'mobilenetv2'},
'name': 'mobilenetv2'}]
If the output description shows it is a MobileNetV2 model, then we can continue to next step to send our request.
6.3 Sending request and get the result¶
Run predict function to send the request, select between TOP1_CLASS and TOP5_CLASS to check the output:
[9]:
# set image path and output strategy(TOP1_CLASS or TOP5_CLASS)
image_path = "/root/airplane.jpg"
strategy = "TOP1_CLASS"
# predict(image_path, servable_name, dataset_name, strategy)
if server_started() is True:
img_viewer = ImageViewer(Image.open(image_path), image_path)
img_viewer.show()
print(predict(image_path, 'mobilenetv2', 'cifar10', strategy))
else:
print("Server not started")
TOP1: airplane, score: 0.22268821299076080322
Check output¶
If user can see output similar to this:
TOP1: airplane, score: 0.22268821299076080322
that means the prediction result is returned and the inference is completed.
## Shutdown server
To restart and try another checkpoint file, click Kernel at the top, then Restart & Clear Output, replace the servable_json code and predict() function
Run the following code to shutdown Flask server:
[10]:
shutdown()
[10]:
'Server shutting down...'
TinyMS SSD300 Tutorial¶
In this tutorial, using TinyMS API to train/serve an SSD300 model will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of six parts, constructing the model, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import json
import time
import tinyms as ts
import xml.etree.ElementTree as et
from PIL import Image
from tinyms import context, layers, primitives as P, Tensor
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.data import VOCDataset, download_dataset
from tinyms.vision import voc_transform, coco_eval, ImageViewer
from tinyms.model import Model, ssd300_mobilenetv2, ssd300_infer
from tinyms.losses import net_with_loss
from tinyms.optimizers import Momentum
from tinyms.callbacks import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor
from tinyms.utils.train.lr_generator import mobilenetv2_lr as ssd300_lr
from tinyms.initializers import initializer, TruncatedNormal
[WARNING] ME(14174:140126738016064,MainProcess):2021-03-19-15:33:42.136.028 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Construct the model¶
[2]:
# build network
net = ssd300_mobilenetv2(class_num=21)
2. Download dataset¶
The VOC dataset will be downloaded if voc folder didn’t exist at the root. If voc folder already exists, this step will not be performed.
[3]:
# download the dataset
voc_path = '/root/voc'
if not os.path.exists(voc_path):
download_dataset('voc', '/root')
print('************Download complete*************')
else:
print('************Dataset already exists.**************')
************** Downloading the VOC2007 dataset **************
[████████████████████████████████████████████████████████████████████████████████████████████████████] 100.00%
============== /root/voc/VOCtrainval_06-Nov-2007.tar is ready ==============
************Download complete*************
3. Train the model & evaluation¶
The dataset for both training and evaluation will be defined here, and the parameters for training also set in this block. A trained ckpt file will be saved to /etc/tinyms/serving/ssd300 folder for later use, meanwhile the evaluation will be performed and the Accuracy can be checked
Notice: Since training SSD300 on CPU is time consuming, we recommend skip training and using provided ckpt files to run.
[ ]:
class TrainingWrapper(layers.Layer):
"""
Encapsulation class of SSD300 network training.
Append an optimizer to the training network after that the construct
function can be called to create the backward graph.
Args:
network (Layer): The training network. Note that loss function should have been added.
optimizer (Optimizer): Optimizer for updating the weights.
sens (float): The adjust parameter. Default: 1.0.
"""
def __init__(self, network, optimizer, sens=1.0):
super(TrainingWrapper, self).__init__(auto_prefix=False)
self.network = network
self.network.set_grad()
self.weights = ts.ParameterTuple(network.trainable_params())
self.optimizer = optimizer
self.grad = P.GradOperation(get_by_list=True, sens_param=True)
self.sens = sens
self.hyper_map = P.HyperMap()
def construct(self, *args):
weights = self.weights
loss = self.network(*args)
sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens)
grads = self.grad(self.network, weights)(*args, sens)
return P.depend(loss, self.optimizer(grads))
def create_voc_label(voc_dir, voc_cls, usage='val'):
"""Get image path and annotation from VOC."""
if not os.path.isdir(voc_dir):
raise ValueError(f'Cannot find {voc_dir} dataset path.')
anno_dir = voc_dir
if os.path.isdir(os.path.join(voc_dir, 'Annotations')):
anno_dir = os.path.join(voc_dir, 'Annotations')
cls_map = {name: i for i, name in enumerate(voc_cls)}
# Fetch the specific xml files path
xml_files = []
with open(os.path.join(voc_dir, 'ImageSets', 'Main', usage+'.txt'), 'r') as f:
for line in f:
xml_files.append(line.strip('\n')+'.xml')
json_dict = {"images": [], "type": "instances", "annotations": [],
"categories": []}
bnd_id = 1
for xml_file in xml_files:
img_id = xml_files.index(xml_file)
tree = et.parse(os.path.join(anno_dir, xml_file))
root_node = tree.getroot()
file_name = root_node.find('filename').text
for obj in root_node.iter('object'):
cls_name = obj.find('name').text
if cls_name not in cls_map:
print(f'Label "{cls_name}" not in "{cls_map}"')
continue
bnd_box = obj.find('bndbox')
x_min = int(float(bnd_box.find('xmin').text)) - 1
y_min = int(float(bnd_box.find('ymin').text)) - 1
x_max = int(float(bnd_box.find('xmax').text)) - 1
y_max = int(float(bnd_box.find('ymax').text)) - 1
o_width = abs(x_max - x_min)
o_height = abs(y_max - y_min)
ann = {'area': o_width * o_height, 'iscrowd': 0,
'image_id': img_id,
'bbox': [x_min, y_min, o_width, o_height],
'category_id': cls_map[cls_name], 'id': bnd_id,
'ignore': 0,
'segmentation': []}
json_dict['annotations'].append(ann)
bnd_id = bnd_id + 1
size = root_node.find("size")
width = int(size.find('width').text)
height = int(size.find('height').text)
image = {'file_name': file_name, 'height': height, 'width': width,
'id': img_id}
json_dict['images'].append(image)
for cls_name, cid in cls_map.items():
cat = {'supercategory': 'none', 'id': cid, 'name': cls_name}
json_dict['categories'].append(cat)
anno_file = os.path.join(anno_dir, 'annotation.json')
with open(anno_file, 'w') as f:
json.dump(json_dict, f)
return anno_file
# check ckpt folder exists or not
ckpt_folder = '/etc/tinyms/serving/ssd300'
ckpt_path = '/etc/tinyms/serving/ssd300/ssd300.ckpt'
if not os.path.exists(ckpt_folder):
!mkdir -p /etc/tinyms/serving/ssd300
else:
print('ssd300 ckpt folder already exists')
# set parameters
epoch_size = 800 # default is 800
batch_size = 32
voc_path = '/root/voc/VOCdevkit/VOC2007'
# set environment parameters
context.set_context(mode=context.GRAPH_MODE, device_target="CPU")
dataset_sink_mode = False
# set dataset parameters
train_dataset = VOCDataset(voc_path, task='Detection', usage='trainval', num_parallel_workers=4, shuffle=True, decode=True)
train_dataset = voc_transform.apply_ds(train_dataset, repeat_size=1, batch_size=batch_size, num_parallel_workers=4, is_training=True)
eval_dataset = VOCDataset(voc_path, task='Detection', usage='val', num_parallel_workers=4, shuffle=True, decode=True)
eval_dataset = voc_transform.apply_ds(eval_dataset, repeat_size=1, batch_size=batch_size, num_parallel_workers=4, is_training=False)
dataset_size = train_dataset.get_dataset_size()
total = eval_dataset.get_dataset_size()
# define the loss function
net = net_with_loss(net)
params = net.trainable_params()
for p in params:
if 'beta' not in p.name and 'gamma' not in p.name and 'bias' not in p.name:
p.set_data(initializer(TruncatedNormal(0.02), p.data.shape, p.data.dtype))
# define the optimizer
pre_trained_epoch_size = 0
save_checkpoint_epochs = 10
lr = 0.01
lr = ssd300_lr(global_step=pre_trained_epoch_size * dataset_size,
lr_init=0.001, lr_end=0.001 * lr, lr_max=lr,
warmup_epochs=2, total_epochs=epoch_size,
steps_per_epoch=dataset_size)
loss_scale = 1.0
opt = Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), lr,0.9, 1.5e-4, loss_scale)
model = Model(TrainingWrapper(net, opt, loss_scale))
model.compile()
ckpoint_cb = ModelCheckpoint(prefix="ssd300", config=CheckpointConfig(
save_checkpoint_steps=save_checkpoint_epochs * dataset_size,
keep_checkpoint_max=10))
print('************************Start training*************************')
model.train(epoch_size, train_dataset, callbacks=[ckpoint_cb, LossMonitor(), TimeMonitor(data_size=dataset_size)],
dataset_sink_mode=dataset_sink_mode)
model.save_checkpoint(ckpt_path)
print('************************Finished training*************************')
eval_net = ssd300_infer(class_num=21)
model = Model(eval_net)
model.load_checkpoint(ckpt_path)
# perform the model predict operation
print("\n========================================\n")
print("total images num: ", total)
print("Processing, please wait a moment...")
start = time.time()
pred_data = []
id_iter = 0
for data in eval_dataset.create_dict_iterator(output_numpy=True):
image_np = data['image']
image_shape = data['image_shape']
output = model.predict(Tensor(image_np))
for batch_idx in range(image_np.shape[0]):
pred_data.append({"boxes": output[0].asnumpy()[batch_idx],
"box_scores": output[1].asnumpy()[batch_idx],
"img_id": id_iter,
"image_shape": image_shape[batch_idx]})
id_iter += 1
cost_time = int((time.time() - start) * 1000)
print(f' 100% [{total}/{total}] cost {cost_time} ms')
# calculate mAP for the predict data
voc_cls = ['background',
'aeroplane', 'bicycle', 'bird', 'boat', 'bottle',
'bus', 'car', 'cat', 'chair', 'cow',
'diningtable', 'dog', 'horse', 'motorbike', 'person',
'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']
anno_file = create_voc_label(voc_path, voc_cls)
mAP = coco_eval(pred_data, anno_file)
print("\n========================================\n")
print(f"mAP: {mAP}")
Notice: If skipped training process, download the pretrained ckpt file and continue to serving
Click HERE to download the ckpt file and save this file to /etc/tinyms/serving/ssd300/ssd300.ckpt.
Or run the following code to download and store the ckpt file:
[4]:
ssd300_ckpt_folder = '/etc/tinyms/serving/ssd300'
ssd300_ckpt_path = '/etc/tinyms/serving/ssd300/ssd300.ckpt'
if not os.path.exists(ssd300_ckpt_folder):
!mkdir -p /etc/tinyms/serving/ssd300
!wget -P /etc/tinyms/serving/ssd300 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/voc/ssd300.ckpt
else:
print('ssd300 ckpt folder already exists')
if not os.path.exists(ssd300_ckpt_path):
!wget -P /etc/tinyms/serving/ssd300 https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/voc/ssd300.ckpt
else:
print('ssd300 ckpt file already exists')
ssd300 ckpt folder already exists
--2021-03-19 15:38:53-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/voc/ssd300.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 49.4.112.90, 121.36.121.44, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 28056511 (27M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/ssd300/ssd300.ckpt’
ssd300.ckpt 100%[===================>] 26.76M 20.7MB/s in 1.3s
2021-03-19 15:38:55 (20.7 MB/s) - ‘/etc/tinyms/serving/ssd300/ssd300.ckpt’ saved [28056511/28056511]
4. Define servable.json¶
Run this code to define the servable json file for later use:
[5]:
servable_json = [{'name': 'ssd300',
'description': 'This servable hosts an ssd300 model predicting bounding boxes',
'model': {
"name": "ssd300",
"format": "ckpt",
"class_num": 21}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
5. Start server¶
5.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. There is a server and client. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which will construct the model, run the prediction and send the result back to the client.
5.2 start server¶
Run the following code block to start the server:
[6]:
start_server()
Server starts at host 127.0.0.1, port 5000
6. Make predictions¶
6.1 Upload the pic¶
A picture is required to be the input. In this tutorial, the SSD300 ckpt requires a picture containing objects of
['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']
Click HERE to download the picture which is used in this tutorial. Upload the pic, if using terminal, either scp or wget will do, if running in Jupyter, click Upload button at the top right and select the picture. Save the picture at the root folder, rename to ssd300_test.jpeg(or any name you want).
Or run the following code to download the picture used in this tutorial:
[7]:
# download the test pic
if not os.path.exists('/root/ssd300_test.jpeg'):
!wget -P /root/ https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/ssd300_test/ssd300_test.jpeg
else:
print('ssd300_test.jpeg already exists')
--2021-03-19 15:38:59-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/tinyms-test-pics/ssd300_test/ssd300_test.jpeg
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 49.4.112.90, 121.36.121.44, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 70412 (69K) [image/jpeg]
Saving to: ‘/root/ssd300_test.jpeg’
ssd300_test.jpeg 100%[===================>] 68.76K 338KB/s in 0.2s
2021-03-19 15:39:00 (338 KB/s) - ‘/root/ssd300_test.jpeg’ saved [70412/70412]
6.2 List servables¶
Now, use list_servables function to check what model is servable right now.
[8]:
list_servables()
[8]:
[{'description': 'This servable hosts an ssd300 model predicting bounding boxes',
'model': {'class_num': 21, 'format': 'ckpt', 'name': 'ssd300'},
'name': 'ssd300'}]
If the output description shows it is an ssd300 model, run the following code to predict the bounding boxes
6.3 Sending request and get the result¶
Run predict function and get the result, right now only TOP1CLASS strategy is supported for SSD300. Call ImageViewer.draw to draw the bounding boxes
[9]:
# set image path and output strategy(only TOP1_CLASS)
image_path = "/root/ssd300_test.jpeg"
strategy = "TOP1_CLASS"
labels = ['background',
'aeroplane', 'bicycle', 'bird', 'boat', 'bottle',
'bus', 'car', 'cat', 'chair', 'cow',
'diningtable', 'dog', 'horse', 'motorbike', 'person',
'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']
# predict(image_path, servable_name, dataset_name, strategy)
# ImageViewer(img, title)
# ImageViewer.draw(predict_result, labels)
if server_started() is True:
res = predict(image_path, 'ssd300', 'voc', strategy)
img_viewer = ImageViewer(Image.open(image_path))
img_viewer.draw(res, labels)
else:
print("Server not started")
Check output¶
If the input picture is shown with bounding boxes labeled with object classes and score, that means the prediction is successfully performed.
TinyMS CycleGAN Tutorial¶
In this tutorial, using TinyMS API to train/serve a CycleGAN model will be demonstrated.
Prerequisite¶
Ubuntu:
18.04Python:
3.7.xFlask:
1.1.2MindSpore:
CPU-1.1.1TinyMS:
0.1.0numpy:
1.17.5Pillow:
8.1.0pip:
21.0.1requests:
2.18.4matplotlib:
3.3.4
Introduction¶
TinyMS is a high-level API which is designed for amateur of deep learning. It minimizes the number of actions of users required to construct, train, evaluate and serve a model. TinyMS also provides tutorials and documentations for developers.
This tutorial consists of five parts, downloading dataset, training, define servable json, starting server and making predictions in which the server will be run in a sub process.
[1]:
import os
import argparse
import json
import tinyms as ts
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
from tinyms import context, Tensor
from tinyms.serving import start_server, predict, list_servables, shutdown, server_started
from tinyms.data import GeneratorDataset, UnalignedDataset, GanImageFolderDataset, DistributedSampler
from tinyms.vision import cyclegan_transform
from tinyms.model.cycle_gan.cycle_gan import get_generator_discriminator, cycle_gan, TrainOneStepG, TrainOneStepD
from tinyms.losses import CycleGANDiscriminatorLoss, CycleGANGeneratorLoss
from tinyms.optimizers import Adam
from tinyms.data.utils import save_image, generate_image_list
from tinyms.utils.common_utils import GanReporter, gan_load_ckpt, GanImagePool
from tinyms.utils.train import cyclegan_lr
from tinyms.utils.eval import CityScapes, fast_hist, get_scores
[WARNING] ME(25552:140556571072320,MainProcess):2021-03-21-15:01:26.554.568 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
1. Download dataset¶
In this tutorial, the cityscapes dataset is used and processed. Click the link before proceeding to the official website to submit and download the dataset.
2. Train the model & evaluation¶
Define parameters and training process
Notice: Training on CPU is time consuming, we recommend skip training and using provided ckpt files to run.
[ ]:
def parse_args():
parser = argparse.ArgumentParser(description='MindSpore Cycle GAN Example')
parser.add_argument('--device_target', type=str, default="CPU", choices=['Ascend', 'GPU', 'CPU'],
help='device where the code will be implemented (default: CPU)')
parser.add_argument('--dataset_path', type=str, default="/root/dataset/cityscapes", help='cityscape dataset path.')
parser.add_argument('--phase', type=str, default="train", help='train, eval or predict.')
parser.add_argument('--model', type=str, default="resnet", choices=("resnet", "unet"),
help='generator model, should be in [resnet, unet].')
parser.add_argument('--max_epoch', type=int, default=200, help='epoch size for training, default is 200.')
parser.add_argument('--n_epoch', type=int, default=100,
help='number of epochs with the initial learning rate, default is 100')
parser.add_argument('--batch_size', type=int, default=1, help='Batch size.')
parser.add_argument("--save_checkpoint_epochs", type=int, default=10,
help="Save checkpoint epochs, default is 10.")
parser.add_argument("--G_A_ckpt", type=str, default="/etc/tinyms/serving/cyclegan_cityscape/G_A.ckpt", help="pretrained checkpoint file path of G_A.")
parser.add_argument("--G_B_ckpt", type=str, default="/etc/tinyms/serving/cyclegan_cityscape/G_B.ckpt", help="pretrained checkpoint file path of G_B.")
parser.add_argument("--D_A_ckpt", type=str, default=None, help="pretrained checkpoint file path of D_A.")
parser.add_argument("--D_B_ckpt", type=str, default=None, help="pretrained checkpoint file path of D_B.")
parser.add_argument('--outputs_dir', type=str, default='/root/',
help='models are saved here, default is ./outputs.')
parser.add_argument('--save_imgs', type=bool, default=True,
help='whether save imgs when epoch end, default is True.')
parser.add_argument("--cityscapes_dir", type=str, default="/root/dataset/cityscapes/testA", help="Path to the original cityscapes dataset")
parser.add_argument("--result_dir", type=str, default="/root/dataset/cityscapes/testB", help="Path to the generated images to be evaluated")
args_opt = parser.parse_args(args=[])
return args_opt
args_opt = parse_args()
context.set_context(mode=context.GRAPH_MODE, device_target="CPU")
dataset_path = args_opt.dataset_path
phase = args_opt.phase
G_A_ckpt = args_opt.G_A_ckpt
G_B_ckpt = args_opt.G_B_ckpt
repeat_size = 1
model = args_opt.model
batch_size = args_opt.batch_size
max_dataset_size = float("inf")
outputs_dir = args_opt.outputs_dir
max_epoch = args_opt.max_epoch
n_epoch = args_opt.n_epoch
n_epoch = min(max_epoch, n_epoch)
def create_dataset(dataset_path, batch_size=1, repeat_size=1, max_dataset_size=None,
shuffle=True, num_parallel_workers=1, phase='train', data_dir='testA'):
""" create Mnist dataset for train or eval.
Args:
data_path: Data path
batch_size: The number of data records in each group
repeat_size: The number of replicated data records
num_parallel_workers: The number of parallel workers
"""
# define dataset and apply the transform func
if phase == 'train':
ds = UnalignedDataset(dataset_path, phase, max_dataset_size=max_dataset_size, shuffle=True)
device_num = 1
distributed_sampler = DistributedSampler(len(ds), num_replicas=device_num, rank=0, shuffle=shuffle)
gan_generator_ds = GeneratorDataset(ds, column_names=["image_A", "image_B"], sampler=distributed_sampler,
num_parallel_workers=num_parallel_workers)
else:
datadir = os.path.join(dataset_path, data_dir)
ds = GanImageFolderDataset(datadir, max_dataset_size=max_dataset_size)
gan_generator_ds = GeneratorDataset(ds, column_names=["image", "image_name"],
num_parallel_workers=num_parallel_workers)
gan_generator_ds = cyclegan_transform.apply_ds(gan_generator_ds,
repeat_size=repeat_size,
batch_size=batch_size,
num_parallel_workers=num_parallel_workers,
shuffle=shuffle,
phase=phase)
dataset_size = len(ds)
return gan_generator_ds, dataset_size
# create dataset
dataset, args_opt.dataset_size = create_dataset(dataset_path, batch_size=batch_size, repeat_size=1,
max_dataset_size=max_dataset_size, shuffle=True,
num_parallel_workers=1,
phase="train",
data_dir=None)
G_A, G_B, D_A, D_B = get_generator_discriminator(model)
gan_load_ckpt(args_opt.G_A_ckpt, args_opt.G_B_ckpt, args_opt.D_A_ckpt, args_opt.D_B_ckpt,
G_A, G_B, D_A, D_B)
generator_net = cycle_gan(G_A, G_B)
# define loss function and optimizer
loss_D = CycleGANDiscriminatorLoss(D_A, D_B)
loss_G = CycleGANGeneratorLoss(generator_net, D_A, D_B)
lr = cyclegan_lr(max_epoch, n_epoch, args_opt.dataset_size)
optimizer_G = Adam(generator_net.trainable_params(),
cyclegan_lr(max_epoch, n_epoch, args_opt.dataset_size), beta1=0.5)
optimizer_D = Adam(loss_D.trainable_params(),
cyclegan_lr(max_epoch, n_epoch, args_opt.dataset_size), beta1=0.5)
# build two net: generator net and descriminator net
net_G = TrainOneStepG(loss_G, generator_net, optimizer_G)
net_D = TrainOneStepD(loss_D, optimizer_D)
# train process
imgae_pool_A = GanImagePool(pool_size=50)
imgae_pool_B = GanImagePool(pool_size=50)
def train_process(args_opt, data_loader, net_G, net_D, imgae_pool_A, imgae_pool_B):
reporter = GanReporter(args_opt)
reporter.info('==========start training===============')
for _ in range(max_epoch):
reporter.epoch_start()
for data in data_loader:
img_A = data["image_A"]
img_B = data["image_B"]
res_G = net_G(img_A, img_B)
fake_A = res_G[0]
fake_B = res_G[1]
res_D = net_D(img_A, img_B, imgae_pool_A.query(fake_A), imgae_pool_B.query(fake_B))
reporter.step_end(res_G, res_D)
reporter.visualizer(img_A, img_B, fake_A, fake_B)
reporter.epoch_end(net_G)
reporter.info('==========end training===============')
data_loader = dataset.create_dict_iterator()
train_process(args_opt, data_loader, net_G, net_D, imgae_pool_A, imgae_pool_B)
# eval
# original image dir
cityscapes_dir = args_opt.cityscapes_dir
# fake image dir generated after predict
result_dir = args_opt.result_dir
def eval_process(args_opt, cityscapes_dir, result_dir):
CS = CityScapes()
hist_perframe = ts.zeros((CS.class_num, CS.class_num)).asnumpy()
cityscapes = generate_image_list(cityscapes_dir)
args_opt.dataset_size = len(cityscapes)
reporter = GanReporter(args_opt)
reporter.start_eval()
for i, img_path in enumerate(cityscapes):
if i % 100 == 0:
reporter.info('Evaluating: %d/%d' % (i, len(cityscapes)))
img_name = os.path.split(img_path)[1]
ids1 = CS.get_id(os.path.join(cityscapes_dir, img_name))
ids2 = CS.get_id(os.path.join(result_dir, img_name))
hist_perframe += fast_hist(ids1.flatten(), ids2.flatten(), CS.class_num)
mean_pixel_acc, mean_class_acc, mean_class_iou, per_class_acc, per_class_iou = get_scores(hist_perframe)
reporter.info("mean_pixel_acc:{}, mean_class_acc:{}, mean_class_iou: {}".format(mean_pixel_acc,
mean_class_acc,
mean_class_iou))
reporter.info("************ Per class numbers below ************")
for i, cl in enumerate(CS.classes):
while len(cl) < 15:
cl = cl + ' '
reporter.info("{}: acc = {}, iou = {}".format(cl, per_class_acc[i], per_class_iou[i]))
reporter.end_eval()
# Compare the similarity between the original image and the fake image
eval_process(args_opt, cityscapes_dir, result_dir)
Notice: If skipped training process, download the pretrained ckpt file and continue to serving
Click G_A to download G_A.ckpt file and G_B to download G_B.ckpt file. Save them to /etc/tinyms/serving/cyclegan_cityscape/
Or run the following code to download and store the ckpt files:
[2]:
ckpt_folder = '/etc/tinyms/serving/cyclegan_cityscape'
G_A_ckpt_path = '/etc/tinyms/serving/cyclegan_cityscape/G_A.ckpt'
G_B_ckpt_path = '/etc/tinyms/serving/cyclegan_cityscape/G_B.ckpt'
if not os.path.exists(ckpt_folder):
!mkdir -p /etc/tinyms/serving/cyclegan_cityscape
!wget -P /etc/tinyms/serving/cyclegan_cityscape https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_A.ckpt
!wget -P /etc/tinyms/serving/cyclegan_cityscape https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_B.ckpt
else:
print('ckpt folder already exists')
if not os.path.exists(G_A_ckpt_path):
!wget -P /etc/tinyms/serving/cyclegan_cityscape https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_A.ckpt
if not os.path.exists(G_B_ckpt_path):
!wget -P /etc/tinyms/serving/cyclegan_cityscape https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_B.ckpt
ckpt folder already exists
--2021-03-21 15:01:30-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_A.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7785480 (7.4M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/cyclegan_cityscape/G_A.ckpt’
G_A.ckpt 100%[===================>] 7.42M 3.58MB/s in 2.1s
2021-03-21 15:01:33 (3.58 MB/s) - ‘/etc/tinyms/serving/cyclegan_cityscape/G_A.ckpt’ saved [7785480/7785480]
--2021-03-21 15:01:34-- https://ascend-tutorials.obs.cn-north-4.myhuaweicloud.com/ckpt_files/cityscapes/G_B.ckpt
Resolving ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)... 49.4.112.113, 121.36.121.44, 49.4.112.5, ...
Connecting to ascend-tutorials.obs.cn-north-4.myhuaweicloud.com (ascend-tutorials.obs.cn-north-4.myhuaweicloud.com)|49.4.112.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7785480 (7.4M) [binary/octet-stream]
Saving to: ‘/etc/tinyms/serving/cyclegan_cityscape/G_B.ckpt’
G_B.ckpt 100%[===================>] 7.42M 9.62MB/s in 0.8s
2021-03-21 15:01:36 (9.62 MB/s) - ‘/etc/tinyms/serving/cyclegan_cityscape/G_B.ckpt’ saved [7785480/7785480]
3. Define servable.json¶
Define the servable json file for model name, format and number of classes for later use.
[3]:
servable_json = [{'name': 'cyclegan_cityscape',
'description': 'This servable hosts a Cycle GAN model predicting for cityscape dataset',
'model': {
"name": "cycle_gan",
"format": "ckpt",
"g_model": "resnet"}}]
os.chdir("/etc/tinyms/serving")
json_data = json.dumps(servable_json, indent=4)
with open('servable.json', 'w') as json_file:
json_file.write(json_data)
4. Start server¶
4.1 Introduction¶
TinyMS Serving is a C/S(client/server) structure. There is a server and client. TinyMS using Flask which is a micro web framework written in python as the C/S communication tool. In order to serve a model, user must start server first. If successfully started, the server will be run in a subprocess and listening to POST requests from 127.0.0.1 port 5000 sent by client and handle the requests using MindSpore backend which will construct the model, run the prediction and send the result back to the client.
4.2 start server¶
Run the following code block to start the server:
[4]:
start_server()
Server starts at host 127.0.0.1, port 5000
5. Make predictions¶
5.1 List servables¶
Now, we can use list_servables function to check what model is servable right now.
[5]:
list_servables()
[5]:
[{'description': 'This servable hosts a Cycle GAN model predicting for cityscape dataset',
'model': {'format': 'ckpt', 'g_model': 'resnet', 'name': 'cycle_gan'},
'name': 'cyclegan_cityscape'}]
If the output description shows it is a CycleGAN model, then we can continue to next step to send our request.
5.2 Sending request and get the result¶
Run predict function to send the request, in this tutorial, both gray to color and color to gray will be demonstrated. Recommend using pics from cityscapes/test to run the predict. If user using own pics, resize to 256*256.
[6]:
servable_name = 'cyclegan_cityscape'
dataset_name = 'cityscape'
if server_started() is True:
# gray to color
testA_path = '/root/dataset/cityscapes/testA/1.jpg'
strategy = 'gray2color'
fakeB_data = predict(testA_path, servable_name, dataset_name, strategy)
# color to gray
testB_path = '/root/dataset/cityscapes/testB/1.jpg'
strategy = 'color2gray'
fakeA_data = predict(testB_path, servable_name, dataset_name, strategy)
# draw the plot
plt.figure(dpi=160, figsize=(10, 10))
plt.subplot(221)
plt.imshow(Image.open(testA_path))
plt.axis('off')
plt.title(testA_path)
plt.subplot(222)
plt.imshow(Image.fromarray(np.uint8(fakeB_data)))
plt.axis('off')
plt.title("fakeB.jpg")
plt.subplot(223)
plt.imshow(Image.open(testB_path))
plt.axis('off')
plt.title(testB_path)
plt.subplot(224)
plt.imshow(Image.fromarray(np.uint8(fakeA_data)))
plt.axis('off')
plt.title("fakeA.jpg")
plt.show()
else:
print("Server not started")
Check output¶
If user can see 4 pics, that means two on the left column are from original test datset, while two pic on the right column are the generated pic.
## Shutdown server
To restart and try another checkpoint file, click Kernel at the top, then Restart & Clear Output, replace the servable_json code and predict() function
Run the following code to shutdown Flask server:
[7]:
shutdown()
[7]:
'Server shutting down...'
TinyMS Online Video Tutorial¶
In order to facilitate the learning of machine learning beginners, TinyMS provides a full set of online video tutorials, which will be updated simultaneously with the iteration of each version. We update and archive according to the content of each episode, including but not limited to video URLs, learning documents and Q&A. If you encounter any questions during the learning process, welcome to add the WeChat of the assistant (Wechat ID: mindspore0328, annotation: TinyMS). The assistant will add you to the TinyMS course learning group, and communicate with developers in the group in time. Of course, we also welcome you to directly pull Issue when you encounter problems, which can also help other developers who encounter the same problem.
Design Concept¶
Background¶
In recent years, with the rapid development of AI technology, deep learning frameworks such as TensorFlow, PyTorch, Apache MXNet and MindSpore have emerged. These frameworks are very good at solving problems for academic research or commercial production, however for first time beginners or application developers with limited deep learning knowledge, much simpler APIs are desired. Alongside the existing efforts like Keras for TensorFlow and Fastai for PyTorch to address the issue, the TinyMS project is a new addition in this field to provide simple high level APIs, tiny runtime footprint, modular development and agile deployment. TinyMS begins with initial focus on MindSpore integration and looks forward to more framework adaptations in the long run.
Interestingly enough, MindSpore’s high-level and mid-level Python APIs have already implemented most of the functions of Keras, and also been on par with Fastai’s design for PyTorch’s flexibility. Therefore unlike Keras and Fastai which are developed with most of their goals to compensate the underlying frameworks, TinyMS is designed to further enhance the experience of the framework, especially for all scenario development in the case of MindSpore.
With the help of TinyMS, the following goals should be achieved:
Quicker to learn: Get started with AI application development in one minute
Easier to develop: Complete the task of changing AI models and dataset from one to the other in one hour
Architecture¶
Design goals of TinyMS:
High level API that are extremely simple to learn and use.
Support complete AI development workflow from data preparation to model training/inference and finally deployment.
Decoupled modules that are could be easily extended.
Small runtime footprint that could be used on mobile, edge or cloud.
Standardizing spec for model training script format.
TinyMS Architecture
Workflow analysis¶
Typical model development workflow:
Data Acquisition: Dataset download, decompression, loading, etc.
Data Processing: Data preprocessing (enhancement) operations performed on the original dataset for better model performance.
Model Construction: Construction of the network, and also the definition of loss function, optimizer, etc.
Model Training: The process of model training, including the definition of callbacks
Accuracy Verification: The process of model accuracy verification, including the definition of metrics
Model Deployment: Model application services via an inference server
TinyMS Workflow
Module design¶
TinyMS has the following modules:
| Name | Introduction | Example Code |
|---|---|---|
| data | Dataset Loading and Downloading | from tinyms.data import MnistDataset, download_dataset |
| hub | Pre-trained Model Hub for Inference and Transfer Learning | from tinyms import hub |
| model | Model High Level API and Predefined Network | from tinyms.model import Model, lenet5 |
| serving | Model Serving | from tinyms.serving import predict |
| vision | Computer Vision Related Data Processing | from tinyms.vision import mnist_transform, Resize |
| text | Natural Language Processing Related Data Processing | from tinyms.text import Lookup |
| callbacks | Callbacks During Model Training | from tinyms.callbacks import ModelCheckpoint |
| common | Basic Components Including Tensor, Numpy Style Functions | from tinyms import Tensor, array |
| context | Global Context | from tinyms import context |
| initializers | Ops Weight Initialization | from tinyms.initializers import Normal |
| layers | Neural Network Layer | from tinyms.layers import Layer, Conv2d |
| losses | Loss Function | from tinyms.losses import SoftmaxCrossEntropyWithLogits |
| metrics | Metrics For Model Verification | from tinyms.metrics import Accuracy |
| optimizers | Optimizer | from tinyms.optimizers import Momentum |
| primitives | Basic Ops | from tinyms.primitives import Add, tensor_add |
Implementation¶
Data loading (data)¶
The data loading module is mainly divided into two parts: dataset download and loading. Through TinyMS’s data loading API, developers can complete the entire process of downloading, decompressing, formatting, and loading common datasets with just two lines of code.
Most AI frameworks do not provide an interface for dataset download. Users need to prepare the dataset in advance, and at the same time, adjust the format (training/validation data set division, etc.) according to the data loading API format provided by the framework itself. To make dataset API usage much more easier, TinyMS provides the download_dataset interface, which supports users to complete the download, decompression and format adjustment operations of the data set with one click; use Mnist dataset as an example:
from tinyms.data import download_dataset
mnist_path = download_dataset('mnist', local_path='./')
For data loading operations, TinyMS completely inherits MindSpore’s native Data Loading API. Users can use the xxxDataset interface to instantiate different data sets very conveniently. Take MnistDataset as an example:
from tinyms.data import MnistDataset
mnist_ds = MnistDataset(mnist_path, shuffle=True)
Data preprocessing (vision, text)¶
Usually in the model development workflow, data processing presents a big challenge: insufficient data, heavy manual labeling task, irregular data format and many other issues. Any of them could affect the network accuracy after training. Most frameworks provide data processing modules. Take MindSpore as an example, it currently provides data processing functions for common scenarios such as CV and NLP (for relevant interface definitions, please refer to mindspore.dataset.vision and mindspore.dataset.text), the user can directly call the preset data processing operator to process pictures or text, and then construct a data processing pipeline to efficiently parallelize massive data (see here).
TinyMS has made further abstraction and encapsulation on the basis of MindSpore, and directly corresponds to the processing of the dataset itself through the DatasetTransform interface, allowing users to utilize a single piece of data or the entire dataset with just one line of code regarding preprocessing operation; take MnistTransform as an example:
from PIL import Image
from tinyms.vision import mnist_transform
# Preprocessing a single one picture
img = mnist_transform(Image.open('picture.jpg'))
# Apply preprocessing to MnistDataset class instance
mnist_ds = mnist_transform.apply_ds(mnist_ds)
Model construction (model)¶
As the core of deep learning model development, the framework’s main responsibility is to provide complete operator expressions to build different network structures. Therefore, the interfaces at the framework level focus more on functional completeness and flexibility, whereas ModelZoo is provided for application development. TinyMS encapsulates the relevant network call API on the ModelZoo script; take the LeNet5 network as an example:
from tinyms.model import lenet5
net = lenet5(class_num=10)
In addition to encapsulating the commonly used network structures, TinyMS also provides a Model high-level API interface (based on MindSpore Model interface package), by drawing on the design idea of Keras Model interface, it not only improves the original API functionalities, but also provides a consistent development experience for Keras users who wants to try TinyMS:
from tinyms.model import Model
model = Model(net)
model.compile(loss_fn=net_loss, optimizer=net_opt)
Model training (losses, optimizers, callbacks)¶
For the model training phase, the most important factors are the definitions of loss functions, optimizers, and callback functions. For beginners, it is not difficult to understand the basic principles of loss functions and optimizers, but a strong mathematical background is required to understand the principles of implementation. Therefore, the TinyMS high-level API encapsulates the loss function and optimizer at the network level, so that users can complete the initialization work with one line of code whether they are training simple or complex networks; take the LeNet5 network as an example:
from tinyms.losses import SoftmaxCrossEntropyWithLogits
from tinyms.optimizers import Momentum
lr = 0.01
momentum = 0.9
net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
net_opt = Momentum(net.trainable_params(), lr, momentum)
Regarding the definition of callback functions, in addition to commonly used callback functions (such as TimeMonitor, LossMonitor, etc.), MindSpore itself provides Callback interface to facilitate user-defined callback functions. The TinyMS high-level API also provides network-level encapsulation, so that users can complete the initialization of the callback function with one line of code; take the MobileNetV2 network as an example:
from tinyms.callbacks import mobilenetv2_cb
net_cb = mobilenetv2_cb(device_target, lr, is_saving_checkpoint, save_checkpoint_epochs, step_size)
Model evaluating (metrics)¶
Model accuracy verification is an indispensable process to verify whether the model accuracy meets the SOTA criteria. MindSpore natively provides measurement interfaces for indicators such as Accuracy and Precision (see here), while providing users with a custom measurement interface Metric. In terms of metric measurement, TinyMS directly inherits the native MindSpore API:
from tinyms.model import Model
from tinyms.metrics import Accuracy
model = Model(net)
model.compile(metrics={"Accuracy": Accuracy())
Pre-trained model loading (hub)¶
TinyMS Hub is a pre-trained model application tool, serving as a channel for model developers and application developers.
Provide model developers with a convenient and fast channel for model release and submission.
Provide application developers with high-quality pre-trained models, and complete the work of model migration to deployment quickly using model loading and fine-tuning APIs.
Current pre-trained models in TinyMS Hub mainly cover four mainstream task scenarios including image classification, object detection, semantic segmentation and recommendation.
There are several of scenarios for users to leverage hub to easily load the pre-trained model:
Load pre-trained model
from PIL import Image from tinyms import hub from tinyms.vision import mnist_transform from tinyms.model import Model img = Image.open(img_path) img = mnist_transform(img) # load LeNet5 pre-trained model net= hub.load('tinyms/0.2/lenet5_v1_mnist', class_num=10) model = Model(net) res = model.predict(ts.expand_dims(ts.array(img), 0)).asnumpy() print("The label is:", mnist_transform.postprocess(res))
Load model checkpoint
from tinyms import hub from tinyms.model import lenet5 from tinyms.utils.train import load_checkpoint ckpt_dist_file = '/tmp/lenet5.ckpt' hub.load_checkpoint('tinyms/0.2/lenet5_v1_mnist', ckpt_dist_file) net = lenet5() load_checkpoint(ckpt_dist_file, net=net)
Load model weights
from tinyms import hub from tinyms.model import lenet5 from tinyms.utils.train import load_param_into_net param_dict = hub.load_weights('tinyms/0.2/lenet5_v1_mnist') net = lenet5() load_param_into_net(net, param_dict)
Model deployment (serving)¶
Model deployment refers to the process of servicing pre-trained models so that they can quickly and efficiently process data input by users and obtain results. MindSpore provides the predict function for inference. TinyMS provides a complete set of start server (start_server), check backend (list_servables), check start status (server_started) and shut down the server (shutdown) and other functions based on Flask ; Take the LeNet5 network as an example:
from tinyms.serving import Server, Client
server = Server()
# Start prediction server
server.start_server()
client = Client()
# List all servables available
client.list_servables()
# Call predict interface
client.predict(image_path, 'lenet5', dataset_name='mnist')
# Shutdown the prediction server
server.shutdown()
tinyms¶
Top-level reference to dtype of common module. This module also provides Numpy-like interfaces in TinyMS.
Examples
>>> import tinyms as ts
>>>
>>> print(ts.ones([2, 3]))
[[1. 1. 1.]
[1. 1. 1.]]
-
tinyms.dtype_to_nptype(type_)[source]¶ Convert MindSpore dtype to numpy data type.
- Parameters
type_ (
mindspore.dtype) – MindSpore’s dtype.- Returns
The data type of numpy.
-
tinyms.issubclass_(type_, dtype)[source]¶ Determine whether type_ is a subclass of dtype.
- Parameters
type_ (
mindspore.dtype) – Target MindSpore dtype.dtype (
mindspore.dtype) – Compare MindSpore dtype.
- Returns
bool, True or False.
-
tinyms.dtype_to_pytype(type_)[source]¶ Convert MindSpore dtype to python data type.
- Parameters
type_ (
mindspore.dtype) – MindSpore’s dtype.- Returns
Type of python.
-
tinyms.pytype_to_dtype(obj)[source]¶ Convert python type to MindSpore type.
- Parameters
obj (type) – A python type object.
- Returns
Type of MindSpore type.
-
tinyms.get_py_obj_dtype(obj)[source]¶ Get the MindSpore data type which corresponds to python type or variable.
- Parameters
obj (type) – An object of python type, or a variable in python type.
- Returns
Type of MindSpore type.
-
class
tinyms.Tensor(input_data=None, dtype=None, shape=None, init=None)[source]¶ Tensor is used for data storage.
Tensor inherits tensor object in C++. Some functions are implemented in C++ and some functions are implemented in Python.
- Parameters
input_data (Tensor, float, int, bool, tuple, list, numpy.ndarray) – Input data of the tensor.
dtype (
mindspore.dtype) – Input data should be None, bool or numeric type defined in mindspore.dtype. The argument is used to define the data type of the output tensor. If it is None, the data type of the output tensor will be as same as the input_data. Default: None.shape (Union[tuple, list, int]) – A list of integers, a tuple of integers or an integer as the shape of output. If input_data is available, shape doesn’t need to be set. Default: None.
init (Initializer) – the information of init data. ‘init’ is used for delayed initialization in parallel mode. Usually, it is not recommended to use ‘init’ interface to initialize parameters in other conditions. If ‘init’ interface is used to initialize parameters, the init_data API need to be called to convert Tensor to the actual data.
- Outputs:
Tensor, with the same shape as input_data.
Examples
>>> import numpy as np >>> import mindspore as ms >>> from mindspore.common.tensor import Tensor >>> from mindspore.common.initializer import One >>> # initialize a tensor with input data >>> t1 = Tensor(np.zeros([1, 2, 3]), ms.float32) >>> assert isinstance(t1, Tensor) >>> assert t1.shape == (1, 2, 3) >>> assert t1.dtype == ms.float32 >>> >>> # initialize a tensor with a float scalar >>> t2 = Tensor(0.1) >>> assert isinstance(t2, Tensor) >>> assert t2.dtype == ms.float64 ... >>> # initialize a tensor with init >>> t3 = Tensor(shape = (1, 3), dtype=ms.float32, init=One()) >>> assert isinstance(t3, Tensor) >>> assert t3.shape == (1, 3) >>> assert t3.dtype == ms.float32
-
property
T¶ Return the transposed tensor.
-
all(axis=(), keep_dims=False)[source]¶ Check all array elements along a given axis evaluate to True.
-
assign_value(self: mindspore._c_expression.Tensor, arg0: mindspore._c_expression.Tensor) → mindspore._c_expression.Tensor¶ Assign another tensor value to this.
- Arg:
value (
mindspore.tensor): The value tensor.
Examples
>>> data = mindspore.Tensor(np.ones((1, 2), np.float32)) >>> data2 = mindspore.Tensor(np.ones((2, 2), np.float32)) >>> data.assign_value(data2) >>> data.shape (2, 2)
-
astype(dtype, copy=True)[source]¶ Return a copy of the tensor, casted to a specified type.
- Parameters
dtype (Union[
mindspore.dtype, str]) – Designated tensor dtype, can be in format ofmindspore.dtype.float32or float32. Default:mindspore.dtype.float32.copy (bool, optional) – By default, astype always returns a newly allocated tensor. If this is set to false, the input tensor is returned instead of a copy if possible. Default: True.
- Returns
Tensor, with the designated dtype.
-
data_sync(self: mindspore._c_expression.Tensor, arg0: bool) → None¶
-
dim(self: mindspore._c_expression.Tensor) → int¶ Get tensor’s data dimension.
- Returns
int, the dimension of tensor.
Examples
>>> data = mindspore.Tensor(np.ones((2, 3))) >>> data.dim() 2
-
property
dtype¶ Return the dtype of the tensor (
mindspore.dtype).
-
expand_as(x)[source]¶ Expand the dimension of target tensor to the dimension of input tensor.
- Parameters
x (Tensor) – The input tensor. The shape of input tensor must obey the broadcasting rule.
- Returns
Tensor, has the same dimension as input tensor.
-
flatten(order='C')[source]¶ Return a copy of the tensor collapsed into one dimension.
- Parameters
order (str, optional) – Can choose between ‘C’ and ‘F’. ‘C’ means to flatten in row-major (C-style) order. ‘F’ means to flatten in column-major (Fortran-style) order. Only ‘C’ and ‘F’ are supported. Default: ‘C’.
- Returns
Tensor, has the same data type as input.
-
property
has_init¶ tensor is inited.
-
init_data(slice_index=None, shape=None, opt_shard_group=None)[source]¶ Get the tensor format data of this Tensor. The init_data function can be called once for the same tensor.
- Parameters
slice_index (int) – Slice index of a parameter’s slices. It is used when initialize a slice of a parameter, it guarantees that devices using the same slice can generate the same tensor.
shape (list(int)) – Shape of the slice, it is used when initialize a slice of the parameter.
opt_shard_group (str) – Optimizer shard group which is used in auto or semi auto parallel mode to get one shard of a parameter’s slice.
-
is_init(self: mindspore._c_expression.Tensor) → bool¶ Get tensor init_flag.
- Returns
bool, whether the tensor init.
Examples
>>> data = mindspore.Tensor(np.ones((2, 3))) >>> data.is_init() False
-
property
itemsize¶ Return the length of one tensor element in bytes.
-
mean(axis=(), keep_dims=False)[source]¶ Reduce a dimension of a tensor by averaging all elements in the dimension.
-
property
nbytes¶ Return the total number of bytes taken by the tensor.
-
property
ndim¶ Return the number of tensor dimensions.
-
ravel()[source]¶ Return a contiguous flattened tensor.
- Returns
Tensor, a 1-D tensor, containing the same elements of the input.
-
reshape(*shape)[source]¶ Give a new shape to a tensor without changing its data.
- Parameters
shape (Union[int, tuple(int), list(int)]) – The new shape should be compatible with the original shape. If an integer, then the result will be a 1-D array of that length. One shape dimension can be -1. In this case, the value is inferred from the length of the array and remaining dimensions.
- Returns
Tensor, with new specified shape.
-
set_cast_dtype(self: mindspore._c_expression.Tensor, dtype: mindspore::Type = None) → None¶
-
set_dtype(self: mindspore._c_expression.Tensor, arg0: mindspore::Type) → mindspore::Type¶ Set the tensor’s data type.
- Arg:
dtype (
mindspore.dtype): The type of output tensor.
Examples
>>> data = mindspore.Tensor(np.ones((1, 2), np.float32)) >>> data.set_dtype(mindspore.int32) mindspore.int32
-
set_init_flag(self: mindspore._c_expression.Tensor, arg0: bool) → None¶ Set tensor init_flag.
Examples
>>> data = mindspore.Tensor(np.ones((2, 3))) >>> data.set_init_flag(True)
-
property
shape¶ Returns the shape of the tensor as a tuple.
-
property
size¶ Returns the total number of elements in tensor.
-
property
strides¶ Return the tuple of bytes to step in each dimension when traversing a tensor.
-
transpose(*axes)[source]¶ Return a view of the tensor with axes transposed.
For a 1-D tensor this has no effect, as a transposed vector is simply the same vector. For a 2-D tensor, this is a standard matrix transpose. For a n-D tensor, if axes are given, their order indicates how the axes are permuted. If axes are not provided and tensor.shape = (i[0], i[1],…i[n-2], i[n-1]), then tensor.transpose().shape = (i[n-1], i[n-2], … i[1], i[0]).
- Parameters
axes (Union[None, tuple(int), list(int), int], optional) – If axes is None or blank, tensor.transpose() will reverse the order of the axes. If axes is tuple(int) or list(int), tensor.transpose() will transpose the tensor to the new axes order. If axes is int, this form is simply intended as a convenience alternative to the tuple/list form.
- Returns
Tensor, has the same dimension as input tensor, with axes suitably permuted.
-
property
virtual_flag¶ Mark tensor is virtual.
-
class
tinyms.RowTensor(indices, values, dense_shape)[source]¶ A sparse representation of a set of tensor slices at given indices.
An RowTensor is typically used to represent a subset of a larger tensor dense of shape [L0, D1, .. , DN] where L0 >> D0.
The values in indices are the indices in the first dimension of the slices that have been extracted from the larger tensor.
The dense tensor dense represented by an RowTensor slices has dense[slices.indices[i], :, :, :, …] = slices.values[i, :, :, :, …].
RowTensor can only be used in the Cell’s construct method.
It is not supported in pynative mode at the moment.
- Parameters
- Returns
RowTensor, composed of indices, values, and dense_shape.
Examples
>>> import mindspore as ms >>> import mindspore.nn as nn >>> class Net(nn.Cell): ... def __init__(self, dense_shape): ... super(Net, self).__init__() ... self.dense_shape = dense_shape ... def construct(self, indices, values): ... x = RowTensor(indices, values, self.dense_shape) ... return x.values, x.indices, x.dense_shape >>> >>> indices = Tensor([0]) >>> values = Tensor([[1, 2]], dtype=ms.float32) >>> out = Net((3, 2))(indices, values) >>> print(out[0]) [[1. 2.]] >>> print(out[1]) [0] >>> print(out[2]) (3, 2)
-
class
tinyms.SparseTensor(indices, values, dense_shape)[source]¶ A sparse representation of a set of nonzero elememts from a tensor at given indices.
SparseTensor can only be used in the Cell’s construct method.
Pynative mode not supported at the moment.
For a tensor dense, its SparseTensor(indices, values, dense_shape) has dense[indices[i]] = values[i].
- Parameters
indices (Tensor) – A 2-D integer Tensor of shape [N, ndims], where N and ndims are the number of values and number of dimensions in the SparseTensor, respectively.
values (Tensor) – A 1-D tensor of any type and shape [N], which supplies the values for each element in indices.
dense_shape (tuple(int)) – A integer tuple of size ndims, which specifies the dense_shape of the sparse tensor.
- Returns
SparseTensor, composed of indices, values, and dense_shape.
Examples
>>> import mindspore as ms >>> import mindspore.nn as nn >>> class Net(nn.Cell): ... def __init__(self, dense_shape): ... super(Net, self).__init__() ... self.dense_shape = dense_shape ... def construct(self, indices, values): ... x = SparseTensor(indices, values, self.dense_shape) ... return x.values, x.indices, x.dense_shape >>> >>> indices = Tensor([[0, 1], [1, 2]]) >>> values = Tensor([1, 2], dtype=ms.float32) >>> out = Net((3, 4))(indices, values) >>> print(out[0]) [1. 2.] >>> print(out[1]) [[0 1] [1 2]] >>> print(out[2]) (3, 4)
-
tinyms.ms_function(fn=None, obj=None, input_signature=None)[source]¶ Create a callable MindSpore graph from a python function.
This allows the MindSpore runtime to apply optimizations based on graph.
- Parameters
fn (Function) – The Python function that will be run as a graph. Default: None.
obj (Object) – The Python Object that provides the information for identifying the compiled function.Default: None.
input_signature (Tensor) – The Tensor which describes the input arguments. The shape and dtype of the Tensor will be supplied to this function. If input_signature is specified, each input to fn must be a Tensor. And the input parameters of fn cannot accept **kwargs. The shape and dtype of actual inputs should keep the same as input_signature. Otherwise, TypeError will be raised. Default: None.
- Returns
Function, if fn is not None, returns a callable function that will execute the compiled function; If fn is None, returns a decorator and when this decorator invokes with a single fn argument, the callable function is equal to the case when fn is not None.
Examples
>>> from mindspore.ops import functional as F ... >>> x = Tensor(np.ones([1, 1, 3, 3]).astype(np.float32)) >>> y = Tensor(np.ones([1, 1, 3, 3]).astype(np.float32)) ... >>> # create a callable MindSpore graph by calling ms_function >>> def tensor_add(x, y): ... z = x + y ... return z ... >>> tensor_add_graph = ms_function(fn=tensor_add) >>> out = tensor_add_graph(x, y) ... >>> # create a callable MindSpore graph through decorator @ms_function >>> @ms_function ... def tensor_add_with_dec(x, y): ... z = x + y ... return z ... >>> out = tensor_add_with_dec(x, y) ... >>> # create a callable MindSpore graph through decorator @ms_function with input_signature parameter >>> @ms_function(input_signature=(Tensor(np.ones([1, 1, 3, 3]).astype(np.float32)), ... Tensor(np.ones([1, 1, 3, 3]).astype(np.float32)))) ... def tensor_add_with_sig(x, y): ... z = x + y ... return z ... >>> out = tensor_add_with_sig(x, y)
-
class
tinyms.Parameter(default_input, name=None, requires_grad=True, layerwise_parallel=False, parallel_optimizer=True)[source]¶ Parameter types of cell models.
After initialized Parameter is a subtype of Tensor.
In auto_parallel mode of “semi_auto_parallel” and “auto_parallel”, if init Parameter by an Tensor, the type of Parameter will be Tensor. Tensor will save the shape and type info of a tensor with no memory usage. The shape can be changed while compiling for auto-parallel. Call init_data will return a Tensor Parameter with initialized data.
Note
Each parameter of Cell is represented by Parameter class. A Parameter has to belong to a Cell. If there is an operator in the network that requires part of the inputs to be Parameter, then the Parameters as this part of the inputs are not allowed to be cast. It is recommended to use the default value of name when initialize a parameter as one attribute of a cell, otherwise, the parameter name may be different than expected.
- Parameters
default_input (Union[Tensor, int, float, numpy.ndarray, list]) – Parameter data, to be set initialized.
name (str) – Name of the child parameter. Default: None.
requires_grad (bool) – True if the parameter requires gradient. Default: True.
layerwise_parallel (bool) – When layerwise_parallel is true in data parallel mode, broadcast and gradients communication would not be applied to parameters. Default: False.
parallel_optimizer (bool) – It is used to filter the weight shard operation in semi auto or auto parallel mode. It works only when enable parallel optimizer in mindspore.context.set_auto_parallel_context(). Default: True.
Examples
>>> from mindspore import Parameter, Tensor >>> from mindspore.common import initializer as init >>> from mindspore.ops import operations as P >>> from mindspore.nn import Cell >>> import mindspore >>> import numpy as np >>> from mindspore import context >>> >>> class Net(Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.matmul = P.MatMul() ... self.weight = Parameter(Tensor(np.ones((1, 2)), mindspore.float32), name="w", requires_grad=True) ... ... def construct(self, x): ... out = self.matmul(self.weight, x) ... return out >>> net = Net() >>> x = Tensor(np.ones((2, 1)), mindspore.float32) >>> print(net(x)) [[2.]] >>> _ = net.weight.set_data(Tensor(np.zeros((1, 2)), mindspore.float32)) >>> print(net(x)) [[0.]]
-
asnumpy(self: mindspore._c_expression.Tensor) → array¶ Convert tensor to numpy.ndarray.
- Returns
numpy.ndarray.
Examples
>>> data = mindspore.Tensor(np.ones((2, 3))) >>> array = data.asnumpy() >>> array array([[1., 1., 1.], [1., 1., 1.]])
-
assign_value(self: mindspore._c_expression.Tensor, arg0: mindspore._c_expression.Tensor) → mindspore._c_expression.Tensor¶ Assign another tensor value to this.
- Arg:
value (
mindspore.tensor): The value tensor.
Examples
>>> data = mindspore.Tensor(np.ones((1, 2), np.float32)) >>> data2 = mindspore.Tensor(np.ones((2, 2), np.float32)) >>> data.assign_value(data2) >>> data.shape (2, 2)
-
property
cache_enable¶ Return whether the parameter is cache enable.
-
property
cache_shape¶ Return the cache shape corresponding to the parameter if use cache.
-
clone(init='same')[source]¶ Clone the parameter.
- Parameters
init (Union[Tensor, str, numbers.Number]) – Initialize the shape of the parameter. Default: ‘same’.
- Returns
Parameter, a new parameter.
-
property
comm_fusion¶ Get the fusion type for communication operators corresponding to this parameter.
-
data_sync(self: mindspore._c_expression.Tensor, arg0: bool) → None¶
-
dim(self: mindspore._c_expression.Tensor) → int¶ Get tensor’s data dimension.
- Returns
int, the dimension of tensor.
Examples
>>> data = mindspore.Tensor(np.ones((2, 3))) >>> data.dim() 2
-
property
dtype¶ Get the MetaTensor’s dtype.
-
from_numpy(self: array) → mindspore._c_expression.Tensor¶ Creates a Tensor from a numpy.ndarray without copy.
- Arg:
array (numpy.ndarray): The input ndarray.
- Returns
Tensor, tensor with shared data to input ndarray.
Examples
>>> a = np.ones((2, 3)) >>> t = mindspore.Tensor.from_numpy(a)
-
init_data(layout=None, set_sliced=False)[source]¶ Initialize the parameter data.
- Parameters
layout (Union[None, list(list(int))]) –
Parameter slice layout [dev_mat, tensor_map, slice_shape]. Default: None.
dev_mat (list(int)): Device matrix.
tensor_map (list(int)): Tensor map.
slice_shape (list(int)): Shape of slice.
set_sliced (bool) – True if the parameter is set sliced after initializing the data. Default: False.
- Raises
RuntimeError – If it is from Initializer, and parallel mode has changed after the Initializer created.
- Returns
Parameter, the Parameter after initializing data. If current Parameter was already initialized before, returns the same initialized Parameter.
-
property
inited_param¶ Get the new parameter after call the init_data.
Default is a None, If self is a Parameter with out data, after call the init_data the initialized Parameter with data will be recorded here.
-
property
is_init¶ Get the initialization status of the parameter.
In GE backend, the Parameter need a “init graph” to sync the data from host to device. This flag indicates whether the data as been sync to the device.
This flag only work in GE, and it will be set to False in other backend.
-
property
name¶ Get the name of the parameter.
-
property
parallel_optimizer¶ Return whether the parameter requires weight shard for parallel optimizer.
-
property
requires_grad¶ Return whether the parameter requires gradient.
-
set_cast_dtype(self: mindspore._c_expression.Tensor, dtype: mindspore::Type = None) → None¶
-
set_dtype(self: mindspore._c_expression.Tensor, arg0: mindspore::Type) → mindspore::Type¶ Set the tensor’s data type.
- Arg:
dtype (
mindspore.dtype): The type of output tensor.
Examples
>>> data = mindspore.Tensor(np.ones((1, 2), np.float32)) >>> data.set_dtype(mindspore.int32) mindspore.int32
-
set_init_flag(self: mindspore._c_expression.Tensor, arg0: bool) → None¶ Set tensor init_flag.
Examples
>>> data = mindspore.Tensor(np.ones((2, 3))) >>> data.set_init_flag(True)
-
set_param_ps(init_in_server=False)[source]¶ Set whether the trainable parameter is updated by parameter server and whether the trainable parameter is initialized on server.
Note
It only works when a running task is in the parameter server mode.
- Parameters
init_in_server (bool) – Whether trainable parameter updated by parameter server is initialized on server. Default: False.
-
property
shape¶ Get the MetaTensor’s shape.
-
property
sliced¶ Get slice status of the parameter.
-
property
unique¶ whether the parameter is already unique or not.
-
class
tinyms.ParameterTuple[source]¶ Class for storing tuple of parameters.
Note
It is used to store the parameters of the network into the parameter tuple collection.
-
count()¶ Return number of occurrences of value.
-
index()¶ Return first index of value.
Raises ValueError if the value is not present.
-
-
tinyms.set_seed(seed)[source]¶ Set global random seed.
Note
The global seed is used by numpy.random, mindspore.common.Initializer, mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution.
If global seed is not set, these packages will use their own default seed independently, numpy.random and mindspore.common.Initializer will choose a random seed, mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution will use zero.
Seed set by numpy.random.seed() only used by numpy.random, while seed set by this API will also used by numpy.random, so just set all seed by this API is recommended.
- Parameters
seed (int) – The seed to be set.
- Raises
ValueError – If seed is invalid (< 0).
TypeError – If seed isn’t a int.
Examples
>>> from mindspore.ops import composite as C >>> from mindspore import Tensor >>> >>> # Note: (1) Please make sure the code is running in PYNATIVE MODE; >>> # (2) Because Composite-level ops need parameters to be Tensors, for below examples, >>> # when using C.uniform operator, minval and maxval are initialised as: >>> minval = Tensor(1.0, ms.float32) >>> maxval = Tensor(2.0, ms.float32) >>> >>> # 1. If global seed is not set, numpy.random and initializer will choose a random seed: >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A1 >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A2 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W1 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W2 >>> # Rerun the program will get different results: >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A3 >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A4 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W3 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W4 >>> >>> # 2. If global seed is set, numpy.random and initializer will use it: >>> set_seed(1234) >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A1 >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A2 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W1 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W2 >>> # Rerun the program will get the same results: >>> set_seed(1234) >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A1 >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A2 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W1 >>> w1 = Parameter(initializer("uniform", [2, 2], ms.float32), name="w1") # W2 >>> >>> # 3. If neither global seed nor op seed is set, mindspore.ops.composite.random_ops and >>> # mindspore.nn.probability.distribution will choose a random seed: >>> c1 = C.uniform((1, 4), minval, maxval) # C1 >>> c2 = C.uniform((1, 4), minval, maxval) # C2 >>> # Rerun the program will get different results: >>> c1 = C.uniform((1, 4), minval, maxval) # C3 >>> c2 = C.uniform((1, 4), minval, maxval) # C4 >>> >>> # 4. If global seed is set, but op seed is not set, mindspore.ops.composite.random_ops and >>> # mindspore.nn.probability.distribution will calculate a seed according to global seed and >>> # default op seed. Each call will change the default op seed, thus each call get different >>> # results. >>> set_seed(1234) >>> c1 = C.uniform((1, 4), minval, maxval) # C1 >>> c2 = C.uniform((1, 4), minval, maxval) # C2 >>> # Rerun the program will get the same results: >>> set_seed(1234) >>> c1 = C.uniform((1, 4), minval, maxval) # C1 >>> c2 = C.uniform((1, 4), minval, maxval) # C2 >>> >>> # 5. If both global seed and op seed are set, mindspore.ops.composite.random_ops and >>> # mindspore.nn.probability.distribution will calculate a seed according to global seed and >>> # op seed counter. Each call will change the op seed counter, thus each call get different >>> # results. >>> set_seed(1234) >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # C2 >>> # Rerun the program will get the same results: >>> set_seed(1234) >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # C2 >>> >>> # 6. If op seed is set but global seed is not set, 0 will be used as global seed. Then >>> # mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution act as in >>> # condition 5. >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # C2 >>> # Rerun the program will get the same results: >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # C2 >>> >>> # 7. Recall set_seed() in the program will reset numpy seed and op seed counter of >>> # mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution. >>> set_seed(1234) >>> np_1 = np.random.normal(0, 1, [1]).astype(np.float32) # A1 >>> c1 = C.uniform((1, 4), minval, maxval, seed=2) # C1 >>> set_seed(1234) >>> np_2 = np.random.normal(0, 1, [1]).astype(np.float32) # still get A1 >>> c2 = C.uniform((1, 4), minval, maxval, seed=2) # still get C1
-
tinyms.abs(x, dtype=None)¶ Calculates the absolute value element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. Currently the backend kernel only supports float calculation, if the input is not a float, then it will be casted to
mstype.float32and casted back.- Parameters
x (Tensor) – Tensor to be used for calculation.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor.
- Raises
TypeError – If input arguments have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([1, 2, 3, -4, -5], np.float32) >>> output = np.absolute(x) >>> print(output) [1. 2. 3. 4. 5.]
-
tinyms.absolute(x, dtype=None)[source]¶ Calculates the absolute value element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. Currently the backend kernel only supports float calculation, if the input is not a float, then it will be casted to
mstype.float32and casted back.- Parameters
x (Tensor) – Tensor to be used for calculation.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor.
- Raises
TypeError – If input arguments have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([1, 2, 3, -4, -5], np.float32) >>> output = np.absolute(x) >>> print(output) [1. 2. 3. 4. 5.]
-
tinyms.add(x1, x2, dtype=None)[source]¶ Adds arguments element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the sum of x1 and x2, element-wise. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.full((3, 2), [1, 2]) >>> x2 = np.full((3, 2), [3, 4]) >>> output = np.add(x1, x2) >>> print(output) [[4 6] [4 6] [4 6]]
-
tinyms.amax(a, axis=None, keepdims=False, initial=None, where=True)[source]¶ Returns the maximum of an array or maximum along an axis.
Note
Numpy argument out is not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
a (Tensor) – Input data.
axis (None or int or tuple of ints, optional) – defaults to None. Axis or axes along which to operate. By default, flattened input is used. If this is a tuple of ints, the maximum is selected over multiple axes, instead of a single axis or all the axes as before.
keepdims (boolean, optional) – defaults to False. If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.
initial (scalar, optional) – The minimum value of an output element. Must be present to allow computation on empty slice.
where (boolean Tensor, optional) – defaults to True. A boolean array which is broadcasted to match the dimensions of array, and selects elements to include in the reduction. If non-default value is passed, initial must also be provided.
- Returns
Tensor or scalar, maximum of a. If axis is None, the result is a scalar value. If axis is given, the result is an array of dimension
a.ndim - 1.- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.arange(4).reshape((2,2)).astype('float32') >>> output = np.amax(a) >>> print(output) 3.0 >>> output = np.amax(a, axis=0) >>> print(output) [2. 3.] >>> output = np.amax(a, axis=1) >>> print(output) [1. 3.] >>> output = np.amax(a, where=np.array([False, True]), initial=-1, axis=0) >>> print(output) [-1. 3.]
-
tinyms.amin(a, axis=None, keepdims=False, initial=None, where=True)[source]¶ Returns the minimum of an array or minimum along an axis.
Note
Numpy argument out is not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
a (Tensor) – Input data.
axis (None or int or tuple of ints, optional) – defaults to None. Axis or axes along which to operate. By default, flattened input is used. If this is a tuple of ints, the minimum is selected over multiple axes, instead of a single axis or all the axes as before.
keepdims (boolean, optional) – defaults to False. If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.
initial (scalar, optional) – The maximum value of an output element. Must be present to allow computation on empty slice.
where (boolean Tensor, optional) – defaults to True. A boolean array which is broadcasted to match the dimensions of array, and selects elements to include in the reduction. If non-default value is passed, initial must also be provided.
- Returns
Tensor or scalar, minimum of a. If axis is None, the result is a scalar value. If axis is given, the result is an array of dimension
a.ndim - 1.- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.arange(4).reshape((2,2)).astype('float32') >>> output = np.amin(a) >>> print(output) 0.0 >>> output = np.amin(a, axis=0) >>> print(output) [0. 1.] >>> output = np.amin(a, axis=1) >>> print(output) [0. 2.] >>> output = np.amin(a, where=np.array([False, True]), initial=10, axis=0) >>> print(output) [10. 1.]
-
tinyms.append(arr, values, axis=None)[source]¶ Appends values to the end of a tensor.
- Parameters
arr (Tensor) – Values are appended to a copy of this tensor.
values (Tensor) – These values are appended to a copy of arr. It must be of the correct shape (the same shape as arr, excluding axis). If axis is not specified, values can be any shape and will be flattened before use.
axis (None, int, optional) – The axis along which values are appended. If axis is not given, both arr and values are flattened before use, default is
None.
- Returns
Tensor, a copy of tensor with values appended to axis.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If specified axis exceeds arr.ndim.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((2, 3)) >>> b = np.ones((2, 1)) >>> print(np.append(a, b, axis=1).shape) (2, 4)
-
tinyms.arange(start, stop=None, step=None, dtype=None)[source]¶ Returns evenly spaced values within a given interval.
- Parameters
start (Union[int, float]) – Start of interval. The interval includes this value. When stop is provided as a position argument, start must be given, when stop is a normal argument, start can be optional, and default is 0. Please see additional examples below.
stop (Union[int, float], optional) – End of interval. The interval does not include this value, except in some cases where step is not an integer and floating point round-off affects the length of out.
step (Union[int, float], optional) – Spacing between values. For any output out, this is the distance between two adjacent values, \(out[i+1] - out[i]\). The default step size is 1. If step is specified as a position argument, start must also be given.
dtype (Union[
mindspore.dtype, str], optional) – Designated tensor dtype. If dtype is None, the data type of the new tensor will be inferred from start, stop and step. Default is None.
- Returns
Tensor with evenly spaced values.
- Raises
TypeError(PyNative Mode) or RuntimeError(Graph Mode) – If input arguments have types not specified above, or arguments are not given in the correct orders specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.arange(0, 5, 1)) [0 1 2 3 4] >>> print(np.arange(3)) [0 1 2] >>> print(np.arange(start=0, stop=3)) [0 1 2] >>> print(np.arange(0, stop=3, step=0.5)) [0. 0.5 1. 1.5 2. 2.5] >>> print(np.arange(stop=3)) # This will lead to TypeError
-
tinyms.arccos(x, dtype=None)[source]¶ Trigonometric inverse cosine, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor. x-coordinate on the unit circle. For real arguments, the domain is \([-1, 1]\).
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor.
- Raises
TypeError – If the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([1, -1], np.float32) >>> output = np.arccos(x) >>> print(output) [0. 3.1415927]
-
tinyms.arccosh(x, dtype=None)[source]¶ Inverse hyperbolic cosine, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(1, 5).astype('float32') >>> print(np.arccosh(x)) [0. 1.316958 1.7627472 2.063437 ]
-
tinyms.arcsin(x, dtype=None)[source]¶ Inverse sine, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor. y-coordinate on the unit circle.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor.
- Raises
TypeError – If the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([1, -1], np.float32) >>> output = np.arcsin(x) >>> print(output) [ 1.5707964 -1.5707964]
-
tinyms.arcsinh(x, dtype=None)[source]¶ Inverse hyperbolic sine element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(5).astype('float32') >>> print(np.arcsinh(x)) [0. 0.8813736 1.4436355 1.8184465 2.0947125]
-
tinyms.arctan(x, dtype=None)[source]¶ Trigonometric inverse tangent, element-wise.
The inverse of tan, so that if \(y = tan(x)\) then \(x = arctan(y)\).
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(5).astype('float32') >>> print(np.tan(x)) [ 0. 1.5574077 -2.1850398 -0.14254655 1.1578213 ]
-
tinyms.arctan2(x1, x2, dtype=None)[source]¶ Element-wise arc tangent of \(x1/x2\) choosing the quadrant correctly.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the sum of x1 and x2, element-wise. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([-1, +1, +1, -1]) >>> x2 = np.array([-1, -1, +1, +1]) >>> output = np.arctan2(x1, x2) >>> print(output) [-2.3561945 2.3561945 0.78539819 -0.78539819]
-
tinyms.arctanh(x, dtype=None)[source]¶ Inverse hyperbolic tangent element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([-0.99, -0.75, -0.5, 0, 0.5]).astype('float32') >>> print(np.arctanh(x)) [-2.646653 -0.97295505 -0.54930615 0. 0.54930615]
-
tinyms.array(obj, dtype=None, copy=True, ndmin=0)[source]¶ Creates a tensor.
This function creates tensors from an array-like object.
- Parameters
obj (Union[int, float, bool, list, tuple]) – Input data, in any form that can be converted to a Tensor. This includes Tensor, list, tuple and numbers.
dtype (Union[
mindspore.dtype, str], optional) – Designated tensor dtype, can be in format of np.int32, or ‘int32’. If dtype isNone, the data type of the new tensor will be inferred from obj. Default isNone.copy (bool) – If True, then the object is copied. Otherwise, a copy will only be made if necessary. Default: True.
ndmin (int) – Specifies the minimum number of dimensions that the resulting tensor should have. Ones will be pre-pended to the shape as needed to meet this requirement. Default: 0
- Returns
Tensor, generated tensor with the specified dtype.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If input obj has different sizes at different dimensions.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.array([1,2,3])) [1 2 3]
-
tinyms.array_split(x, indices_or_sections, axis=0)[source]¶ Splits a tensor into multiple sub-tensors.
Note
Currently, array_split only supports
mindspore.float32onCPU.The only difference between
np.splitandnp.array_splitis thatnp.array_splitallows indices_or_sections to be an integer that does not equally divide the axis. For a tensor of length l that should be split into n sections, it returns \(l % n\) sub-arrays of size \(l//n + 1\) and the rest of size \(l//n\).- Parameters
x (Tensor) – A Tensor to be divided.
indices_or_sections (Union[int, tuple(int), list(int)]) – If integer, \(N\), the tensor will be divided into \(N\) tensors along axis. If tuple(int), list(int) or of sorted integers, the entries indicate where along axis the array is split. For example, \([2, 3]\) would, for \(axis=0\), result in three sub-tensors \(x[:2]\), \(x[2:3]\). If an index exceeds the dimension of the array along axis, an empty sub-array is returned correspondingly.
axis (int) – The axis along which to split. Default: 0.
- Returns
A list of sub-tensors.
- Raises
TypeError – If argument indices_or_sections is not integer, tuple(int) or list(int) or argument axis is not integer.
ValueError – If argument axis is out of range of \([-x.ndim, x.ndim)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> input_x = np.arange(9).astype("float32") >>> output = np.array_split(input_x, 4) >>> print(output) (Tensor(shape=[3], dtype=Float32, value= [ 0.00000000e+00, 1.00000000e+00, 2.00000000e+00]), Tensor(shape=[2], dtype=Float32, value= [ 3.00000000e+00, 4.00000000e+00]), Tensor(shape=[2], dtype=Float32, value= [ 5.00000000e+00, 6.00000000e+00]), Tensor(shape=[2], dtype=Float32, value= [ 7.00000000e+00, 8.00000000e+00]))
-
tinyms.asarray(a, dtype=None)[source]¶ Converts the input to tensor.
This function converts tensors from an array-like object.
- Parameters
a (Union[int, float, bool, list, tuple, Tensor]) – Input data, in any form that can be converted to a Tensor. This includes Tensor, list, tuple and numbers.
dtype (Union[
mindspore.dtype, str], optional) – Designated tensor dtype, can be in format of np.int32, or ‘int32’. If dtype isNone, the data type of the new tensor will be inferred from obj. Default isNone.
- Returns
Tensor, generated tensor with the specified dtype.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If input a has different sizes at different dimensions.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.asarray([1,2,3])) [1 2 3]
-
tinyms.asfarray(a, dtype=mindspore.float32)[source]¶ Similar to asarray, converts the input to a float tensor.
If non-float dtype is defined, this function will return a float32 tensor instead.
- Parameters
a (Union[int, float, bool, list, tuple, Tensor]) –
- Input data, in any form that can
be converted to a Tensor. This includes Tensor, list, tuple and numbers.
- dtype (Union[
mindspore.dtype, str], optional): Designated tensor dtype, can be in format of np.int32, or ‘int32’. If dtype is
None, the data type of the new tensor will be inferred from a. Default ismindspore.float32.
- Returns
Tensor, generated tensor with the specified float dtype.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If input a has different sizes at different dimensions.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.asfarray([1,2,3])) [1. 2. 3.]
-
tinyms.atleast_1d(*arys)[source]¶ Converts inputs to arrays with at least one dimension.
Scalar inputs are converted to 1-dimensional arrays, whilst higher-dimensional inputs are preserved.
Note
In graph mode, returns a tuple of tensor instead of a list of tensors.
- Parameters
*arys (Tensor) – one or more input tensors.
- Returns
Tensor, or list of tensors, each with
a.ndim >= 1.- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((2, 3)) >>> b = np.ones(()) >>> c = np.ones(5) >>> output = np.atleast_1d(a, b, c) >>> print(output) [Tensor(shape=[2, 3], dtype=Float32, value= [[1.00000000e+000, 1.00000000e+000, 1.00000000e+000], [1.00000000e+000, 1.00000000e+000, 1.00000000e+000]]), Tensor(shape=[1], dtype=Float32, value= [1.00000000e+000]), Tensor(shape=[5], dtype=Float32, value= [1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000])]
-
tinyms.atleast_2d(*arys)[source]¶ Reshapes inputs as arrays with at least two dimensions.
Note
In graph mode, returns a tuple of tensor instead of a list of tensors.
- Parameters
*arys (Tensor) – one or more input tensors.
- Returns
Tensor, or list of tensors, each with
a.ndim >= 2.- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((2, 3)) >>> b = np.ones(()) >>> c = np.ones(5) >>> output = np.atleast_2d(a, b, c) >>> print(output) [Tensor(shape=[2, 3], dtype=Float32, value= [[1.00000000e+000, 1.00000000e+000, 1.00000000e+000], [1.00000000e+000, 1.00000000e+000, 1.00000000e+000]]), Tensor(shape=[1, 1], dtype=Float32, value= [[1.00000000e+000]]), Tensor(shape=[1, 5], dtype=Float32, value= [[1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000]])]
-
tinyms.atleast_3d(*arys)[source]¶ Reshapes inputs as arrays with at least three dimensions.
Note
In graph mode, returns a tuple of tensor instead of a list of tensors.
- Parameters
*arys (Tensor) – one or more input tensors.
- Returns
Tensor, or list of tensors, each with
a.ndim >= 3. For example, a 1-D array of shape (N,) becomes a tensor of shape (1, N, 1), and a 2-D array of shape (M, N) becomes a tensor of shape (M, N, 1).- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((2, 3)) >>> b = np.ones(()) >>> c = np.ones(5) >>> output = np.atleast_3d(a, b, c) >>> print(output) [Tensor(shape=[2, 3, 1], dtype=Float32, value= [[[1.00000000e+000], [1.00000000e+000], [1.00000000e+000]], [[1.00000000e+000], [1.00000000e+000], [1.00000000e+000]]]), Tensor(shape=[1, 1, 1], dtype=Float32, value= [[[1.00000000e+000]]]), Tensor(shape=[1, 5, 1], dtype=Float32, value= [[[1.00000000e+000], [1.00000000e+000], [1.00000000e+000], [1.00000000e+000], [1.00000000e+000]]])]
-
tinyms.average(x, axis=None, weights=None, returned=False)[source]¶ Computes the weighted average along the specified axis.
- Parameters
x (Tensor) – A Tensor to be averaged.
axis (Union[None, int, tuple(int)]) – Axis along which to average x. Default: None. If the axis is None, it will average over all of the elements of the tensor x. If the axis is negative, it counts from the last to the first axis.
weights (Union[None, Tensor]) – Weights associated with the values in x. Default: None. If weights is None, all the data in x are assumed to have a weight equal to one. If weights is 1-D tensor, the length must be the same as the given axis. Otherwise, weights should have the same shape as x.
returned (bool) – Default: False. If True, the tuple (average, sum_of_weights) is returned. If False, only the average is returned.
- Returns
Averaged Tensor. If returned is True, return tuple.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> input_x = np.array([[1., 2.], [3., 4.]]) >>> output = np.average(input_x, axis=0, weights=input_x, returned=True) >>> print(output) (Tensor(shape=[2], dtype=Float32, value= [ 2.50000000e+00, 3.33333325e+00]), Tensor(shape=[2], dtype=Float32, value= [ 4.00000000e+00, 6.00000000e+00]))
-
tinyms.broadcast_arrays(*args)[source]¶ Broadcasts any number of arrays against each other.
Note
Numpy argument subok is not supported. In graph mode, returns a tuple of Tensor instead of a list of Tensor.
- Parameters
*args (Tensor) – The arrays to broadcast.
- Returns
List of Tensor.
- Raises
ValueError – if arrays cannot be broadcast.
- Supported Platforms:
AscendGPUCPU
Example
>>> import mindspore.numpy as np >>> x = np.array([[1,2,3]]) >>> y = np.array([[4],[5]]) >>> output = np.broadcast_arrays(x, y) >>> print(output) [Tensor(shape=[2, 3], dtype=Int32, value= [[1, 2, 3], [1, 2, 3]]), Tensor(shape=[2, 3], dtype=Int32, value= [[4, 4, 4], [5, 5, 5]])]
-
tinyms.broadcast_to(array, shape)[source]¶ Broadcasts an array to a new shape.
- Parameters
- Returns
Tensor, original array broadcast to the given shape.
- Raises
ValueError – if array cannot be broadcast to shape.
- Supported Platforms:
AscendGPUCPU
Example
>>> import mindspore.numpy as np >>> x = np.array([1, 2, 3]) >>> output = np.broadcast_to(x, (3, 3)) >>> print(output) [[1 2 3] [1 2 3] [1 2 3]]
-
tinyms.cbrt(x, dtype=None)[source]¶ Returns the cube-root of a tensor, element-wise.
Note
Numpy arguments casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.asarray([1, -1, 3, -8, 64]) >>> output = np.cbrt(a) >>> print(output) [ 1. -1. 1.4422495 -2. 4. ]
-
tinyms.ceil(x, dtype=None)[source]¶ Returns the ceiling of the input, element-wise.
The ceil of the scalar x is the smallest integer i, such that
i >= x.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
x (Tensor) – input values.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, the floor of each element in x. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([-1.7, -1.5, -0.2, 0.2, 1.5, 1.7, 2.0]) >>> output = np.ceil(a) >>> print(output) [-1. -1. -0. 1. 2. 2. 2.]
-
tinyms.clip(x, xmin, xmax, dtype=None)[source]¶ Clips (limits) the values in an array.
Given an interval, values outside the interval are clipped to the interval edges. For example, if an interval of \([0, 1]\) is specified, values smaller than 0 become 0, and values larger than 1 become 1.
- Parameters
x (Tensor) – Tensor containing elements to clip.
xmin (Tensor, scalar, None) – Minimum value. If None, clipping is not performed on lower interval edge. Not more than one of xmin and xmax may be None.
xmax (Tensor, scalar, None) – Maximum value. If None, clipping is not performed on upper interval edge. Not more than one of xmin and xmax may be None. If xmin or xmax are tensors, then the three tensors will be broadcasted to match their shapes.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor, a tensor with the elements of x, but where values < xmin are replaced with xmin, and those > xmax with xmax.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([1, 2, 3, -4, 0, 3, 2, 0]) >>> output = np.clip(x, 0, 2) >>> print(output) [1 2 2 0 0 2 2 0]
-
tinyms.column_stack(tup)[source]¶ Stacks 1-D tensors as columns into a 2-D tensor. 2-D tensors are stacked as-is, like np.hstack.
- Parameters
tup (Union[Tensor, tuple, list]) – A sequence of 1-D or 2-D tensors. All of them must have the same shape except the axis to be concatenated.
- Returns
2-D Tensor, formed by stacking the given tensors.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If tup is not Tensor, list or tuple.
ValueError – If tup is empty.
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([1, 2, 3]).astype('int32') >>> x2 = np.array([4, 5, 6]).astype('int32') >>> output = np.column_stack((x1, x2)) >>> print(output) [[1 4] [2 5] [3 6]]
-
tinyms.concatenate(arrays, axis=0)[source]¶ Joins a sequence of tensors along an existing axis.
Note
To match Numpy behaviour, \(axis >= 32\) will not cause value error, the axis will be treated as
Noneinstead.- Parameters
- Returns
A tensor concatenated from a tensor or a list of tensors.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If axis is not in the range of \([-ndim, ndim-1]\), and less than 32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.ones((1,2,3)) >>> x2 = np.ones((1,2,1)) >>> x = np.concatenate((x1, x2), axis=-1) >>> print(x.shape) (1, 2, 4)
-
tinyms.convolve(a, v, mode='full')[source]¶ Returns the discrete, linear convolution of two one-dimensional sequences.
Note
If v is longer than a, the tensors are swapped before computation.
- Parameters
a (Union[list, tuple, Tensor]) – First one-dimensional input tensor.
v (Union[list, tuple, Tensor]) – Second one-dimensional input tensor.
mode (str, optional) – By default, mode is ‘full’. This returns the convolution at each point of overlap, with an output shape of \((N+M-1,)\). At the end-points of the convolution, the signals do not overlap completely, and boundary effects may be seen. If mode is ‘same’, it returns output of length \(max(M, N)\). Boundary effects are still visible. If mode is ‘valid’, it returns output of length \(max(M, N) - min(M, N) + 1\). The convolution product is only given for points where the signals overlap completely. Values outside the signal boundary have no effect.
- Returns
Tensor, discrete, linear convolution of a and v.
- Raises
TypeError – if the inputs have types not specified above.
ValueError – if a and v are empty or have wrong dimensions
- Supported Platforms:
GPU
Examples
>>> import mindspore.numpy as np >>> output = np.convolve([1., 2., 3., 4., 5.], [2., 3.], mode="valid") >>> print(output) [ 3. 6. 9. 12.]
-
tinyms.cos(x, dtype=None)[source]¶ Cosine element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(5).astype('float32') >>> print(np.cos(x)) [ 1. 0.5403023 -0.41614684 -0.9899925 -0.6536436 ]
-
tinyms.cosh(x, dtype=None)[source]¶ Hyperbolic cosine, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(5).astype('float32') >>> print(np.cosh(x)) [ 1. 1.5430807 3.7621956 10.067662 27.308233 ]
-
tinyms.count_nonzero(x, axis=None, keepdims=False)[source]¶ Counts the number of non-zero values in the tensor x.
- Parameters
x (Tensor) – The tensor for which to count non-zeros.
axis (Union[int,tuple], optional) – Axis or tuple of axes along which to count non-zeros. Default is None, meaning that non-zeros will be counted along a flattened version of x.
keepdims (bool, optional) – If this is set to True, the axes that are counted are left in the result as dimensions with size one. With this option, the result will broadcast correctly against x.
- Returns
Tensor, indicating number of non-zero values in the x along a given axis. Otherwise, the total number of non-zero values in x is returned.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([1, 2, 3, -4, 0, 3, 2, 0]) >>> output = np.count_nonzero(x) >>> print(output) 6
-
tinyms.cov(m, y=None, rowvar=True, bias=False, ddof=None, fweights=None, aweights=None, dtype=None)[source]¶ Estimates a covariance matrix, given data and weights.
Covariance indicates the level to which two variables vary together. If we examine N-dimensional samples, \(X = [x_1, x_2, ... x_N]^T\), then the covariance matrix element \(C_{ij}\) is the covariance of \(x_i\) and \(x_j\). The element \(C_{ii}\) is the variance of \(x_i\).
Note
fweights and aweights must be all positive, in Numpy if negative values are detected, a value error will be raised, in MindSpore we converts all values to positive instead.
- Parameters
m (Union[Tensor, list, tuple]) – A 1-D or 2-D tensor containing multiple variables and observations. Each row of m represents a variable, and each column represents a single observation of all those variables. Also see rowvar below.
y (Union[Tensor, list, tuple], optional) – An additional set of variables and observations. y has the same form as that of m.
rowvar (bool, optional) – If rowvar is
True(default), then each row represents a variable, with observations in the columns. Otherwise, the relationship is transposed: each column represents a variable, while the rows contain observations.bias (bool, optional) – Default normalization (
False) is by \((N - 1)\), where \(N\) is the number of observations given (unbiased estimate). If bias isTrue, then normalization is by N. These values can be overridden by using the keyword ddof.ddof (int, optional) – If not
None, the default value implied by bias is overridden. Note that \(ddof=1\) will return the unbiased estimate, even if both fweights and aweights are specified, and \(ddof=0\) will return the simple average. See the notes for the details. The default value isNone.fweights (Union[Tensor, list, tuple], optional) – 1-D tensor of integer frequency weights; the number of times each observation vector should be repeated.
aweights (Union[Tensor, list, tuple], optional) – 1-D tensor of observation vector weights. These relative weights are typically larger for observations considered more important and smaller for observations considered less important. If \(ddof=0\) the tensor of weights can be used to assign probabilities to observation vectors.
dtype (Union[
mindspore.dtype, str], optional) – Data-type of the result. By default, the return data-type will have mstype.float32 precision.
- Returns
Tensor, the covariance matrix of the variables.
- Raises
TypeError – if the inputs have types not specified above.
ValueError – if m and y have wrong dimensions.
RuntimeError – if aweights and fweights have dimensions > 2.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.cov([[2., 3., 4., 5.], [0., 2., 3., 4.], [7., 8., 9., 10.]]) >>> print(output) [[1.6666666 2.1666667 1.6666666] [2.1666667 2.9166667 2.1666667] [1.6666666 2.1666667 1.6666666]]
-
tinyms.cross(a, b, axisa=-1, axisb=-1, axisc=-1, axis=None)[source]¶ Returns the cross product of two (arrays of) vectors.
The cross product of a and b in \(R^3\) is a vector perpendicular to both a and b. If a and b are arrays of vectors, the vectors are defined by the last axis of a and b by default, and these axes can have dimensions 2 or 3. Where the dimension of either a or b is 2, the third component of the input vector is assumed to be zero and the cross product calculated accordingly. In cases where both input vectors have dimension 2, the z-component of the cross product is returned.
- Parameters
a (Union[int, float, bool, list, tuple, Tensor]) – Components of the first vector(s).
b (Union[int, float, bool, list, tuple, Tensor]) – Components of the second vector(s).
axisa (int, optional) – Axis of a that defines the vector(s). By default, the last axis.
axisb (int, optional) – Axis of b that defines the vector(s). By default, the last axis.
axisc (int, optional) – Axis of c containing the cross product vector(s). Ignored if both input vectors have dimension 2, as the return is scalar. By default, the last axis.
axis (int, optional) – If defined, the axis of a, b and c that defines the vector(s) and cross product(s). Overrides axisa, axisb and axisc.
- Returns
Tensor, vector cross product(s).
- Raises
ValueError – when the dimensions of the vector(s) in a and/or b equal 2 or 3.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([[1,2,3], [4,5,6]]) >>> y = np.array([[4,5,6], [1,2,3]]) >>> output = np.cross(x, y) >>> print(output) [[-3 6 -3] [ 3 -6 3]] >>> output = np.cross(x, y, axisc=0) [[-3 3] [ 6 -6] [-3 3]]
-
tinyms.cumsum(a, axis=None, dtype=None)[source]¶ Returns the cumulative sum of the elements along a given axis.
Note
If
a.dtypeisint8,int16orbool, the result dtype will be elevated toint32.- Parameters
a (Tensor) – Input tensor.
axis (int, optional) – Axis along which the cumulative sum is computed. The default (None) is to compute the cumsum over the flattened array.
dtype (
mindspore.dtype, optional) – If not specified, stay the same as a, unless a has an integer dtype with a precision less than that of the default platform integer. In that case, the default platform integer is used.
- Returns
Tensor.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If axis is out of range.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.cumsum(np.ones((3,3)), axis=0) >>> print(output) [[1. 1. 1.] [2. 2. 2.] [3. 3. 3.]]
-
tinyms.deg2rad(x, dtype=None)[source]¶ Converts angles from degrees to radians.
- Parameters
x (Tensor) – Angles in degrees.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor, the corresponding angle in radians. This is a tensor scalar if x is a tensor scalar.
- Raises
TypeError – if x is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([1, 2, 3, -4, -5]) >>> output = np.deg2rad(x) >>> print(output) [ 0.01745329 0.03490658 0.05235988 -0.06981317 -0.08726647]
-
tinyms.diag(v, k=0)[source]¶ Extracts a diagonal or construct a diagonal array.
- Parameters
- Returns
Tensor, the extracted diagonal or constructed diagonal array.
- Raises
ValueError – if input is not 1-D or 2-D.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(9).reshape((3,3)) >>> print(x) [[0 1 2] [3 4 5] [6 7 8]] >>> output = np.diag(x) >>> print(output) [0 4 8] >>> output = np.diag(x, k=1) >>> print(output) [1 5] >>> output = np.diag(x, k=-1) >>> print(output) [3 7]
-
tinyms.diag_indices(n, ndim=2)[source]¶ Returns the indices to access the main diagonal of an array.
This returns a tuple of indices that can be used to access the main diagonal of an array a with
a.ndim >= 2dimensions and shape (n, n, …, n). Fora.ndim = 2this is the usual diagonal, fora.ndim > 2this is the set of indices to accessa[i, i, ..., i]fori = [0..n-1].- Parameters
- Returns
Tuple of Tensor.
- Raises
TypeError – if input are not integers.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.diag_indices(5, 3) >>> print(output) (Tensor(shape=[5], dtype=Int32, value= [0, 1, 2, 3, 4]), Tensor(shape=[5], dtype=Int32, value= [0, 1, 2, 3, 4]), Tensor(shape=[5], dtype=Int32, value= [0, 1, 2, 3, 4]))
-
tinyms.diagflat(v, k=0)[source]¶ Creates a two-dimensional array with the flattened input as a diagonal.
Note
On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
- Returns
Tensor, The 2-D output array.
- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.diagflat(np.asarray([[1,2], [3,4]])) >>> print(output) [[1 0 0 0] [0 2 0 0] [0 0 3 0] [0 0 0 4]] >>> output = np.diagflat(np.asarray([1,2]), 1) >>> print(output) [[0 1 0] [0 0 2] [0 0 0]]
-
tinyms.diagonal(a, offset=0, axis1=0, axis2=1)[source]¶ Returns specified diagonals.
If a is 2-D, returns the diagonal of a with the given offset, i.e., the collection of elements of the form
a[i, i+offset]. If a has more than two dimensions, then the axes specified by axis1 and axis2 are used to determine the 2-D sub-array whose diagonal is returned. The shape of the resulting array can be determined by removing axis1 and axis2 and appending an index to the right equal to the size of the resulting diagonals.- Parameters
a (Tensor) – Array from which the diagonals are taken.
offset (int, optional) – Offset of the diagonal from the main diagonal. Can be positive or negative. Defaults to main diagonal.
axis1 (int, optional) – Axis to be used as the first axis of the 2-D sub-arrays from which the diagonals should be taken. Defaults to first axis (0).
axis2 (int, optional) – Axis to be used as the second axis of the 2-D sub-arrays from which the diagonals should be taken. Defaults to second axis.
- Returns
Tensor, if a is 2-D, then a 1-D array containing the diagonal. If
a.ndim > 2, then the dimensions specified by axis1 and axis2 are removed, and a new axis inserted at the end corresponding to the diagonal.- Raises
ValueError – if the input tensor has less than two dimensions.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.arange(4).reshape(2,2) >>> print(a) [[0 1] [2 3]] >>> output = np.diagonal(a) >>> print(output) [0 3] >>> output = np.diagonal(a, 1) >>> print(output) [1] >>> a = np.arange(8).reshape(2, 2, 2) >>> print(a) [[[0 1] [2 3]] [[4 5] [6 7]]] >>> output = np.diagonal(a, 0, 0, 1) >>> print(output) [[0 6] [1 7]]
-
tinyms.diff(a, n=1, axis=-1, prepend=None, append=None)[source]¶ Calculates the n-th discrete difference along the given axis.
The first difference is given by \(out[i] = a[i+1] - a[i]\) along the given axis, higher differences are calculated by using diff iteratively.
- Parameters
a (Tensor) – Input tensor.
n (int, optional) – The number of times values are differenced. If zero, the input is returned as-is.
axis (int, optional) – The axis along which the difference is taken, default is the last axis.
prepend/append (Tensor, optional) – Values to prepend or append to a along axis prior to performing the difference. Scalar values are expanded to arrays with length 1 in the direction of axis and the shape of the input array in along all other axes. Otherwise the dimension and shape must match a except along axis.
- Returns
The n-th differences. The shape of the output is the same as a except along axis where the dimension is smaller by n. The type of the output is the same as the type of the difference between any two elements of a. This is the same as the type of a in most cases.
- Raises
TypeError – If inputs have types not specified above.
ValueError – If
n < 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> arr = np.array([1, 3, -1, 0, 4]) >>> print(np.diff(arr, n=2)) [-6 5 3]
-
tinyms.divide(x1, x2, dtype=None)[source]¶ Returns a true division of the inputs, element-wise.
Instead of the Python traditional ‘floor division’, this returns a true division.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, this is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.full((3, 2), [1, 2]) >>> x2 = np.full((3, 2), [3, 4]) >>> output = np.divide(x1, x2) >>> print(output) [[0.33333334 0.5 ] [0.33333334 0.5 ] [0.33333334 0.5 ]]
-
tinyms.divmod(x1, x2, dtype=None)¶ Returns element-wise quotient and remainder simultaneously.
- Parameters
- Returns
Element-wise quotient and remainder from floor division, in format of (quotient, remainder)
- Raises
TypeError – if x1 and x2 are not Tensor or scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([1, 2, 3, 4, 5]) >>> print(np.divmod(a, 1.5)) (Tensor(shape=[5], dtype=Float32, value= [ 0.00000000e+00, 1.00000000e+00, 2.00000000e+00, 2.00000000e+00, 3.00000000e+00]), Tensor(shape=[5], dtype=Float32, value= [ 1.00000000e+00, 5.00000000e-01, 0.00000000e+00, 1.00000000e+00, 5.00000000e-01]))
-
tinyms.dot(a, b)[source]¶ Returns the dot product of two arrays.
Specifically, If both a and b are 1-D arrays, it is inner product of vectors (without complex conjugation). If both a and b are 2-D arrays, it is matrix multiplication. If either a or b is 0-D (scalar), it is equivalent to multiply. If a is an N-D array and b is a 1-D array, it is a sum product over the last axis of a and b. If a is an N-D array and b is an M-D array (where
M>=2), it is a sum product over the last axis of a and the second-to-last axis of b:dot(a, b)[i,j,k,m] = sum(a[i,j,:] * b[k,:,m])Note
Numpy argument out is not supported. On GPU, the supported dtypes are np.float16, and np.float32. On CPU, the supported dtypes are np.float16, np.float32, and np.float64.
- Parameters
- Returns
Tensor or scalar, the dot product of a and b. If a and b are both scalars or both 1-D arrays then a scalar is returned; otherwise an array is returned
- Raises
ValueError – If the last dimension of a is not the same size as the second-to-last dimension of b.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.full((1, 3), 7).astype('float32') >>> b = np.full((2, 3, 4), 5).astype('float32') >>> output = np.dot(a, b) >>> print(output) [[[105. 105. 105. 105.] [105. 105. 105. 105.]]]
-
tinyms.dsplit(x, indices_or_sections)[source]¶ Splits a tensor into multiple sub-tensors along the 3rd axis (depth). It is equivalent to split with \(axis=2\) (default), the array is always split along the third axis regardless of the array dimension.
- Parameters
x (Tensor) – A Tensor to be divided.
indices_or_sections (Union[int, tuple(int), list(int)]) – If integer, \(N\), the tensor will be divided into \(N\) equal tensors along axis. If tuple(int), list(int) or of sorted integers, the entries indicate where along axis the array is split. For example, \([2, 3]\) would, for \(axis=0\), result in three sub-tensors \(x[:2]\), \(x[2:3]\). If an index exceeds the dimension of the array along axis, an empty sub-array is returned correspondingly.
- Returns
A list of sub-tensors.
- Raises
TypeError – If argument indices_or_sections is not integer.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> input_x = np.arange(6).reshape((1, 2, 3)).astype('float32') >>> output = np.dsplit(input_x, 3) >>> print(output) (Tensor(shape=[1, 2, 1], dtype=Float32, value=[[[ 0.00000000e+00], [ 3.00000000e+00]]]), Tensor(shape=[1, 2, 1], dtype=Float32, value=[[[ 1.00000000e+00], [ 4.00000000e+00]]]), Tensor(shape=[1, 2, 1], dtype=Float32, value=[[[ 2.00000000e+00], [ 5.00000000e+00]]]))
-
tinyms.dstack(tup)[source]¶ Stacks tensors in sequence depth wise (along the third axis). This is equivalent to concatenation along the third axis. 1-D tensors \((N,)\) should be reshaped to \((1,N,1)\). 2-D tensors \((M,N)\) should be reshaped to \((M,N,1)\) before concatenation.
- Parameters
tup (Union[Tensor, tuple, list]) – A sequence of tensors. The tensors must have the same shape along all but the third axis. 1-D or 2-D tensors must have the same shape.
- Returns
Stacked Tensor, formed by stacking the given tensors.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If tup is not Tensor, list or tuple.
ValueError – If tup is empty.
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([1, 2, 3]).astype('float32') >>> x2 = np.array([4, 5, 6]).astype('float32') >>> output = np.dstack((x1, x2)) >>> print(output) [[[1. 4.] [2. 5.] [3. 6.]]]
-
tinyms.ediff1d(ary, to_end=None, to_begin=None)[source]¶ The differences between consecutive elements of a tensor.
- Parameters
- Returns
The differences.
- Raises
TypeError – If inputs have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> arr = np.array([1, 3, -1, 0, 4]) >>> print(np.ediff1d(arr)) [ 2 -4 1 4]
-
tinyms.empty(shape, dtype=mindspore.float32)[source]¶ Returns a new array of given shape and type, without initializing entries.
Note
Numpy argument order is not supported. Object arrays are not supported.
- Parameters
- Returns
Tensor, array of uninitialized (arbitrary) data of the given shape and dtype.
- Raises
TypeError – if the input shape or dtype is invalid.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.empty((2, 3)) >>> print(output) # result may vary Tensor(shape=[2, 3], dtype=Float32, value= <uninitialized>)
-
tinyms.empty_like(prototype, dtype=None, shape=None)[source]¶ Returns a new array with the same shape and type as a given array.
Note
Input array must have the same size across a dimension. If prototype is not a Tensor, dtype is float32 by default if not provided.
- Parameters
- Returns
Tensor, array of uninitialized (arbitrary) data with the same shape and type as prototype.
- Raises
ValueError – if prototype is not a Tensor, list or tuple.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((4,1,2)) >>> output = np.empty_like(a) >>> print(output) # result may vary Tensor(shape=[4, 1, 2], dtype=Float32, value= <uninitialized>)
-
tinyms.equal(x1, x2, dtype=None)[source]¶ Returns the truth value of
(x1 == x2)element-wise.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, element-wise comparison of x1 and x2. Typically of type bool, unless dtype is passed. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.equal(np.array([0, 1, 3]), np.arange(3)) >>> print(output) [ True True False]
-
tinyms.exp(x, dtype=None)[source]¶ Calculates the exponential of all elements in the input array.
Note
Numpy arguments casting, order, subok, signature, and extobj are not supported. When where is provided, out must have a tensor value. out is not supported for storing the result, however it can be used in combination with where to set the value at indices for which where is set to False. On GPU, the supported dtypes are np.float16, and np.float32. On CPU, the supported dtypes are np.float16, np.float32, np.float64.
- Parameters
x (Tensor) – input data.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, element-wise exponential of x. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.exp(np.arange(5).astype(np.float32)) >>> print(output) [ 1. 2.718282 7.3890557 20.085537 54.598145 ]
-
tinyms.exp2(x, dtype=None)[source]¶ Calculates
2**pfor all p in the input array.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
x (Tensor) – input values.
dtype (
mindspore.dtype, optional) – defaults toNone. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, element-wise 2 to the power x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([2, 3]).astype(np.float32) >>> output = np.exp2(x) >>> print(output) [4. 8.]
-
tinyms.expand_dims(a, axis)[source]¶ Expands the shape of a tensor.
Inserts a new axis that will appear at the axis position in the expanded tensor shape.
- Parameters
- Returns
Tensor, with the number of dimensions increased at specified axis.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If axis exceeds a.ndim.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((2,2)) >>> x = np.expand_dims(x,0) >>> print(x.shape) (1, 2, 2)
-
tinyms.expm1(x, dtype=None)[source]¶ Calculates
exp(x) - 1for all elements in the array.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16, and np.float32. On CPU, the supported dtypes are np.float16, and np.float32.
- Parameters
x (Tensor) – input data.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, element-wise exponential minus one,
out = exp(x) - 1. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.expm1(np.arange(5).astype(np.float32)) >>> print(output) [ 0. 1.7182819 6.389056 19.085537 53.59815 ]
-
tinyms.eye(N, M=None, k=0, dtype=mindspore.float32)[source]¶ Returns a 2-D tensor with ones on the diagnoal and zeros elsewhere.
- Parameters
N (int) – Number of rows in the output, must be larger than 0.
M (int, optional) – Number of columns in the output. If is
None, defaults to N, if defined, must be larger than 0. Deault isNone.k (int, optional) – Index of the diagonal: 0 (the default) refers to the main diagonal, a positive value refers to an upper diagonal, and a negative value to a lower diagonal. Default is 0.
dtype (Union[
mindspore.dtype, str], optional) – Designated tensor dtype. Default is mstype.float32.
- Returns
A tensor of shape (N, M). A tensor where all elements are equal to zero, except for the k-th diagonal, whose values are equal to one.
- Raises
TypeError – If input arguments have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.eye(2, 2)) [[1. 0.] [0. 1.]]
-
tinyms.fabs(x, dtype=None)¶ Calculates the absolute value element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. Currently the backend kernel only supports float calculation, if the input is not a float, then it will be casted to
mstype.float32and casted back.- Parameters
x (Tensor) – Tensor to be used for calculation.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor.
- Raises
TypeError – If input arguments have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([1, 2, 3, -4, -5], np.float32) >>> output = np.absolute(x) >>> print(output) [1. 2. 3. 4. 5.]
-
tinyms.fix(x)[source]¶ Rounds to nearest integer towards zero.
Rounds an array of floats element-wise to nearest integer towards zero. The rounded values are returned as floats.
Note
Numpy argument out is not supported.
- Parameters
x (Tensor) – An array of floats to be rounded.
- Returns
Tensor.
- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.fix(np.array([2.1, 2.9, -2.1, -2.9])) >>> print(output) [ 2. 2. -2. -2.]
-
tinyms.flip(m, axis=None)[source]¶ Reverses the order of elements in an array along the given axis.
The shape of the array is preserved, but the elements are reordered.
Note
On CPU, the supported dtypes are np.float16, np.float32, and np.float64.
- Parameters
m (Tensor) – Input array.
axis (None or int or tuple of ints, optional) – Axis or axes along which to flip over. The default,
axis=None, will flip over all of the axes of the input array. If axis is negative it counts from the last to the first axis. If axis is a tuple of ints, flipping is performed on all of the axes specified in the tuple.
- Returns
Tensor, with the entries of axis reversed.
- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
GPU
Example
>>> import mindspore.numpy as np >>> A = np.arange(8.0).reshape((2,2,2)) >>> output = np.flip(A) >>> print(output) [[[7, 6], [5, 4]], [[3, 2], [1, 0]]] >>> output = np.flip(A, (0, 2)) >>> print(output) [[[5, 4], [7, 6]], [[1, 0], [3, 2]]]
-
tinyms.fliplr(m)[source]¶ Flips the entries in each row in the left/right direction. Columns are preserved, but appear in a different order than before.
Note
On CPU, the supported dtypes are np.float16, np.float32, and np.float64.
- Parameters
m (Tensor) – Input array.
- Returns
Tensor.
- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
GPU
Example
>>> import mindspore.numpy as np >>> A = np.arange(8.0).reshape((2,2,2)) >>> output = np.fliplr(A) >>> print(output) [[[2., 3.], [0., 1.]], [[6., 7.], [4., 5.]]]
-
tinyms.flipud(m)[source]¶ Flips the entries in each column in the up/down direction. Rows are preserved, but appear in a different order than before.
Note
On CPU, the supported dtypes are np.float16, np.float32, and np.float64.
- Parameters
m (Tensor) – Input array.
- Returns
Tensor.
- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
GPU
Example
>>> import mindspore.numpy as np >>> A = np.arange(8.0).reshape((2,2,2)) >>> output = np.flipud(A) >>> print(output) [[[4., 5.], [6., 7.]], [[0., 1.], [2., 3.]]]
-
tinyms.float_power(x1, x2, dtype=None)[source]¶ First array elements raised to powers from second array, element-wise.
Raise each base in x1 to the positionally-corresponding power in x2. x1 and x2 must be broadcastable to the same shape. This differs from the power function in that integers, float16, and float64 are promoted to floats with a minimum precision of float32 so that the result is always inexact. The intent is that the function will return a usable result for negative powers and seldom overflow for positive powers.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. Integers and floats are promoted to float32 instead of float64.
- Parameters
- Returns
Tensor or scalar, the bases in x1 raised to the exponents in x2. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.arange(6) >>> x2 = np.array(3) >>> output = np.float_power(x1, x2) >>> print(output) [ 0. 1. 8. 27. 64. 125.]
-
tinyms.floor(x, dtype=None)[source]¶ Returns the floor of the input, element-wise.
The floor of the scalar x is the largest integer i, such that
i <= x.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16 and np.float32. On CPU, the supported dtypes are np.float16, np.float32, and np.float64.
- Parameters
x (Tensor) – input data.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, the floor of each element in x. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.floor(np.array([-1.7, -1.5, -0.2, 0.2, 1.5, 1.7, 2.0])) >>> print(output) [-2. -2. -1. 0. 1. 1. 2.]
-
tinyms.floor_divide(x1, x2, dtype=None)[source]¶ Returns the largest integer smaller or equal to the division of the inputs. It is equivalent to the Python // operator and pairs with the Python % (remainder), function so that
a = a % b + b * (a // b)up to roundoff.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.floor_divide(np.array([1., 2., 3., 4.]), np.array(2.5)) >>> print(output) [0. 0. 1. 1.]
-
tinyms.fmod(x1, x2, dtype=None)[source]¶ Returns the element-wise remainder of division.
This is the NumPy implementation of the C library function fmod, the remainder has the same sign as the dividend x1. It is equivalent to the Matlab(TM) rem function and should not be confused with the Python modulus operator
x1 % x2.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the remainder of the division of x1 by x2. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.fmod(np.array([-3, -2, -1, 1, 2, 3]), np.array(2)) >>> print(output) [-1 0 -1 1 0 1]
-
tinyms.full(shape, fill_value, dtype=None)[source]¶ Returns a new tensor of given shape and type, filled with fill_value.
- Parameters
shape (Union[int, tuple(int), list(int)]) – Shape of the new tensor, e.g., \((2, 3)\) or \(2\).
fill_value (Union[int, float, bool, list, tuple]) – Scalar or array_like fill value.
dtype (Union[
mindspore.dtype, str], optional) – Designated tensor dtype, if dtype isNone, the data type of the new tensor will be inferred from fill_value. Default isNone.
- Returns
Tensor, with the designated shape and dtype, filled with fill_value.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If shape has entries < 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.full((2,2), True)) [[True True] [True True]]
-
tinyms.full_like(a, fill_value, dtype=None, shape=None)[source]¶ Returns a full array with the same shape and type as a given array.
Note
Input array must have the same size across a dimension. If a is not a Tensor, dtype is float32 by default if not provided.
- Parameters
a (Union[Tensor, list, tuple]) – The shape and data-type of a define these same attributes of the returned array.
fill_value (scalar) – Fill value.
dtype (
mindspore.dtype, optional) – Overrides the data type of the result.shape (int or sequence of ints, optional) – Overrides the shape of the result.
- Returns
Tensor, array of fill_value with the same shape and type as a.
- Raises
ValueError – if a is not a Tensor, list or tuple.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((4,1,2)) >>> output = np.full_like(a, 0.5) >>> print(output) [[[0.5 0.5]] [[0.5 0.5]] [[0.5 0.5]] [[0.5 0.5]]]
-
tinyms.gcd(x1, x2, dtype=None)[source]¶ Returns the greatest common divisor of
|x1|and|x2|.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the greatest common divisor of the absolute value of the inputs. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.gcd(np.arange(6), np.array(20)) >>> print(output) [20 1 2 1 4 5]
-
tinyms.geomspace(start, stop, num=50, endpoint=True, dtype=None, axis=0)[source]¶ Returns numbers spaced evenly on a log scale (a geometric progression).
This is similar to logspace, but with endpoints specified directly. Each output sample is a constant multiple of the previous.
- Parameters
start (Union[int, list(int), tuple(int), tensor]) – The starting value of the sequence.
stop (Union[int, list(int), tuple(int), tensor]) – The final value of the sequence, unless endpoint is False. In that case, num + 1 values are spaced over the interval in log-space, of which all but the last (a sequence of length num) are returned.
num (int, optional) – Number of samples to generate. Default is 50.
endpoint (bool, optional) – If True, stop is the last sample. Otherwise, it is not included. Default is True.
dtype (Union[
mindspore.dtype, str], optional) – Designated tensor dtype, can be in format of np.float32, or float32.If dtype is None, infer the data type from other input arguments. Default is None.axis (int, optional) – The axis in the result to store the samples. Relevant only if start or stop is array-like. By default (0), the samples will be along a new axis inserted at the beginning. Use -1 to get an axis at the end. Default is 0.
- Returns
Tensor, with samples equally spaced on a log scale.
- Raises
TypeError – If input arguments have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> output = np.geomspace(1, 256, num=9) >>> print(output) [ 1. 2. 4. 8. 16. 32. 64. 128. 256.] >>> output = np.geomspace(1, 256, num=8, endpoint=False) >>> print(output) [ 1. 2. 4. 8. 16. 32. 64. 128.]
-
tinyms.greater(x1, x2, dtype=None)[source]¶ Returns the truth value of
(x1 > x2)element-wise.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, element-wise comparison of x1 and x2. Typically of type bool, unless dtype is passed. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.greater(np.array([4, 2]), np.array([2, 2])) >>> print(output) [ True False]
-
tinyms.greater_equal(x1, x2, dtype=None)[source]¶ Returns the truth value of
(x1 >= x2)element-wise.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, element-wise comparison of x1 and x2. Typically of type bool, unless dtype is passed. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.greater_equal(np.array([4, 2, 1]), np.array([2, 2, 2])) >>> print(output) [ True True False]
-
tinyms.heaviside(x1, x2, dtype=None)[source]¶ Computes the Heaviside step function.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the output array, element-wise Heaviside step function of x1. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.heaviside(np.array([-1.5, 0, 2.0]), np.array(0.5)) >>> print(output) [0. 0.5 1. ] >>> output = np.heaviside(np.array([-1.5, 0, 2.0]), np.array(1)) >>> print(output) [0. 1. 1.]
-
tinyms.hsplit(x, indices_or_sections)[source]¶ Splits a tensor into multiple sub-tensors horizontally (column-wise). It is equivalent to split with \(axis=1\) (default), the array is always split along the second axis regardless of the array dimension.
- Parameters
x (Tensor) – A Tensor to be divided.
indices_or_sections (Union[int, tuple(int), list(int)]) – If integer, \(N\), the tensor will be divided into \(N\) equal tensors along axis. If tuple(int), list(int) or of sorted integers, the entries indicate where along axis the array is split. For example, \([2, 3]\) would, for \(axis=0\), result in three sub-tensors \(x[:2]\), \(x[2:3]\). If an index exceeds the dimension of the array along axis, an empty sub-array is returned correspondingly.
- Returns
A list of sub-tensors.
- Raises
TypeError – If argument indices_or_sections is not integer.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> input_x = np.arange(6).reshape((2, 3)).astype('float32') >>> output = np.hsplit(input_x, 3) >>> print(output) (Tensor(shape=[2, 1], dtype=Float32, value=[[ 0.00000000e+00], [ 3.00000000e+00]]), Tensor(shape=[2, 1], dtype=Float32, value=[[ 1.00000000e+00], [ 4.00000000e+00]]), Tensor(shape=[2, 1], dtype=Float32, value=[[ 2.00000000e+00], [ 5.00000000e+00]]))
-
tinyms.hstack(tup)[source]¶ Stacks tensors in sequence horizontally. This is equivalent to concatenation along the second axis, except for 1-D tensors where it concatenates along the first axis.
- Parameters
tup (Union[Tensor, tuple, list]) – A sequence of 1-D or 2-D tensors. The tensors must have the same shape along all but the second axis, except 1-D tensors which can be any length.
- Returns
Stacked Tensor, formed by stacking the given tensors.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If tup is not Tensor, list or tuple.
ValueError – If tup is empty.
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([1, 2, 3]).astype('float32') >>> x2 = np.array([4, 5, 6]).astype('float32') >>> output = np.hstack((x1, x2)) >>> print(output) [1. 2. 3. 4. 5. 6.]
-
tinyms.hypot(x1, x2, dtype=None)[source]¶ Given the “legs” of a right triangle, returns its hypotenuse.
Equivalent to
sqrt(x1**2 + x2**2), element-wise. If x1 or x2 is scalar_like (i.e., unambiguously cast-able to a scalar type), it is broadcast for use with each element of the other argument. (See Examples)Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16 and np.float32. On CPU, the supported dtypes are np.float16, np.float32, and np.float64.
- Parameters
- Returns
Tensor or scalar, the hypotenuse of the triangle(s). This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.hypot(3*np.ones((3, 3)), 4*np.ones((3, 3))) >>> print(output) [[5. 5. 5.] [5. 5. 5.] [5. 5. 5.]] >>> output = np.hypot(3*np.ones((3, 3)), np.array([4.0])) >>> print(output) [[5. 5. 5.] [5. 5. 5.] [5. 5. 5.]]
-
tinyms.identity(n, dtype=mindspore.float32)[source]¶ Returns the identity tensor.
- Parameters
n (int) – Number of rows and columns in the output, must be larger than 0.
dtype (Union[
mindspore.dtype, str], optional) – Designated tensor dtype, default ismstype.float32.
- Returns
A tensor of shape (n, n), where all elements are equal to zero, except for the diagonal, whose values are equal to one.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If input arguments have types not specified above.
Examples
>>> import mindspore.numpy as np >>> print(np.identity(2)) [[1. 0.] [0. 1.]]
-
tinyms.in1d(ar1, ar2, invert=False)[source]¶ Tests whether each element of a 1-D array is also present in a second array.
Returns a boolean array the same length as ar1 that is True where an element of ar1 is in ar2 and False otherwise.
Note
Numpy argument assume_unique is not supported since the implementation does not rely on the uniqueness of the input arrays.
- Parameters
ar1 (array_like) – Input array with shape (M,).
ar2 (array_like) – The values against which to test each value of ar1.
invert (boolean, optional) – If True, the values in the returned array are inverted (that is, False where an element of ar1 is in ar2 and True otherwise). Default is False.
- Returns
Tensor, with shape (M,). The values
ar1[in1d]are in ar2.
- Supported Platforms:
AscendGPUCPU
Examples
>>> test = np.array([0, 1, 2, 5, 0]) >>> states = [0, 2] >>> mask = np.in1d(test, states) >>> print(mask) [ True False True False True] >>> mask = np.in1d(test, states, invert=True) >>> print(mask) [False True False True False]
-
tinyms.indices(dimensions, dtype=mindspore.int32, sparse=False)[source]¶ Returns an array representing the indices of a grid.
Computes an array where the subarrays contain index values 0, 1, … varying only along the corresponding axis.
- Parameters
dimensions (tuple or list of ints) – The shape of the grid.
dtype (data type, optional) – Data type of the result.
sparse (boolean, optional) – Defaults to False. Return a sparse representation of the grid instead of a dense representation.
- Returns
Tensor or tuple of Tensor, If sparse is False, returns one array of grid indices,
grid.shape = (len(dimensions),) + tuple(dimensions). If sparse is True, returns a tuple of arrays, withgrid[i].shape = (1, ..., 1, dimensions[i], 1, ..., 1)withdimensions[i]in the ith place- Raises
TypeError – if input dimensions is not a tuple or list.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> grid = np.indices((2, 3)) >>> print(grid) [Tensor(shape=[2, 3], dtype=Int32, value= [[0, 0, 0], [1, 1, 1]]), Tensor(shape=[2, 3], dtype=Int32, value= [[0, 1, 2], [0, 1, 2]])]
-
tinyms.inner(a, b)[source]¶ Returns the inner product of two tensors.
Ordinary inner product of vectors for 1-D tensors (without complex conjugation), in higher dimensions a sum product over the last axes.
Note
Numpy argument out is not supported. On GPU, the supported dtypes are np.float16, and np.float32. On CPU, the supported dtypes are np.float16, np.float32, and np.float64.
- Parameters
- Returns
Tensor or scalar.
- Raises
ValueError – if
x1.shape[-1] != x2.shape[-1].
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((5, 3)) >>> b = np.ones((2, 7, 3)) >>> output = np.inner(a, b) >>> print(output) [[[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]] [[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]] [[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]] [[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]] [[3. 3. 3. 3. 3. 3. 3.] [3. 3. 3. 3. 3. 3. 3.]]]
-
tinyms.isclose(a, b, rtol=1e-05, atol=1e-08, equal_nan=False)[source]¶ Returns a boolean tensor where two tensors are element-wise equal within a tolerance.
The tolerance values are positive, typically very small numbers. The relative difference (\(rtol * abs(b)\)) and the absolute difference atol are added together to compare against the absolute difference between a and b.
Note
For finite values, isclose uses the following equation to test whether two floating point values are equivalent. \(absolute(a - b) <= (atol + rtol * absolute(b))\)
- Parameters
a (Union[Tensor, list, tuple]) – Input first tensor to compare.
b (Union[Tensor, list, tuple]) – Input second tensor to compare.
rtol (Number) – The relative tolerance parameter (see Note).
atol (Number) – The absolute tolerance parameter (see Note).
equal_nan (bool) – Whether to compare
NaNas equal. If True,NaNinwill be considered equal to NaN in b in the output tensor. (a) –
- Returns
A
booltensor of where a and b are equal within the given tolerance.- Raises
TypeError – If inputs have types not specified above.
- Supported Platforms:
GPUCPU
Examples
>>> a = np.array([0,1,2,float('inf'),float('inf'),float('nan')]) >>> b = np.array([0,1,-2,float('-inf'),float('inf'),float('nan')]) >>> print(np.isclose(a, b)) [ True True False False True False] >>> print(np.isclose(a, b, equal_nan=True)) [ True True False False True True]
-
tinyms.isfinite(x, dtype=None)[source]¶ Tests element-wise for finiteness (not infinity or not Not a Number).
The result is returned as a boolean array.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
x (Tensor) – Input values.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, true where x is not positive infinity, negative infinity, or NaN; false otherwise. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.isfinite(np.array([np.inf, 1., np.nan]).astype('float32')) >>> print(output) [False True False]
-
tinyms.isin(element, test_elements, invert=False)[source]¶ Calculates element in test_elements, broadcasting over element only. Returns a boolean array of the same shape as element that is True where an element of element is in test_elements and False otherwise.
Note
Numpy argument assume_unique is not supported since the implementation does not rely on the uniqueness of the input arrays.
- Parameters
element (array_like) – Input array.
test_elements (array_like) – The values against which to test each value of element.
invert (boolean, optional) – If True, the values in the returned array are inverted, as if calculating element not in test_elements. Default is False.
- Returns
Tensor, has the same shape as element. The values
element[isin]are in test_elements.
- Supported Platforms:
AscendGPUCPU
Examples
>>> element = 2*np.arange(4).reshape((2, 2)) >>> test_elements = [1, 2, 4, 8] >>> mask = np.isin(element, test_elements) >>> print(mask) [[False True] [ True False]] >>> mask = np.isin(element, test_elements, invert=True) >>> print(mask) [[ True False] [False True]]
-
tinyms.isinf(x, dtype=None)[source]¶ Tests element-wise for positive or negative infinity.
Returns a boolean array of the same shape as x, True where
x == +/-inf, otherwise False.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. Only np.float32 is currently supported.
- Parameters
x (Tensor) – Input values.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, true where x is positive or negative infinity, false otherwise. This is a scalar if x is a scalar.
- Supported Platforms:
GPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.isinf(np.array(np.inf, np.float32)) >>> print(output) True >>> output = np.isinf(np.array([np.inf, -np.inf, 1.0, np.nan], np.float32)) >>> print(output) [ True True False False]
-
tinyms.isnan(x, dtype=None)[source]¶ Tests element-wise for NaN and return result as a boolean array.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. Only np.float32 is currently supported.
- Parameters
x (Tensor) – Input values.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, true where x is NaN, false otherwise. This is a scalar if x is a scalar.
- Supported Platforms:
GPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.isnan(np.array(np.nan, np.float32)) >>> print(output) True >>> output = np.isnan(np.array(np.inf, np.float32)) >>> print(output) False
-
tinyms.isneginf(x)[source]¶ Tests element-wise for negative infinity, returns result as bool array.
Note
Numpy argument out is not supported. Only np.float32 is currently supported.
- Parameters
x (Tensor) – Input values.
- Returns
Tensor or scalar, true where x is negative infinity, false otherwise. This is a scalar if x is a scalar.
- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
GPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.isneginf(np.array([-np.inf, 0., np.inf], np.float32)) >>> print(output) [ True False False]
-
tinyms.isposinf(x)[source]¶ Tests element-wise for positive infinity, returns result as bool array.
Note
Numpy argument out is not supported. Only np.float32 is currently supported.
- Parameters
x (Tensor) – Input values.
- Returns
Tensor or scalar, true where x is positive infinity, false otherwise. This is a scalar if x is a scalar.
- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
GPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.isposinf(np.array([-np.inf, 0., np.inf], np.float32)) >>> print(output) [False False True]
-
tinyms.isscalar(element)[source]¶ Returns True if the type of element is a scalar type.
Note
Only object types recognized by the mindspore parser are supported, which includes objects, types, methods and functions defined within the scope of mindspore. Other built-in types are not supported.
- Parameters
element (any) – Input argument, can be of any type and shape.
- Returns
Boolean, True if element is a scalar type, False if it is not.
- Raises
TypeError – if the type of element is not supported by mindspore parser.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.isscalar(3.1) >>> print(output) True >>> output = np.isscalar(np.array(3.1)) >>> print(output) False >>> output = np.isscalar(False) >>> print(output) True >>> output = np.isscalar('numpy') >>> print(output) True
-
tinyms.ix_(*args)[source]¶ Constructs an open mesh from multiple sequences.
This function takes N 1-D sequences and returns N outputs with N dimensions each, such that the shape is 1 in all but one dimension and the dimension with the non-unit shape value cycles through all N dimensions. Using ix_ one can quickly construct index arrays that will index the cross product.
a[np.ix_([1,3],[2,5])]returns the array[[a[1,2] a[1,5]], [a[3,2] a[3,5]]].Note
Boolean masks are not supported.
- Parameters
*args (Tensor) – 1-D sequences.
- Returns
Tuple of Tensor, N arrays with N dimensions each, with N the number of input sequences. Together these arrays form an open mesh.
- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> ixgrid = np.ix_(np.array([0, 1]), np.array([2, 4])) >>> print(ixgrid) (Tensor(shape=[2, 1], dtype=Int32, value= [[0], [1]]), Tensor(shape=[1, 2], dtype=Int32, value= [[2, 4]]))
-
tinyms.kron(a, b)[source]¶ Kronecker product of two arrays.
Computes the Kronecker product, a composite array made of blocks of the second array scaled by the first.
- Parameters
- Returns
Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.kron([1,10,100], [5,6,7]) >>> print(output) [ 5 6 7 50 60 70 500 600 700] >>> output = np.kron([5,6,7], [1,10,100]) >>> print(output) [ 5 50 500 6 60 600 7 70 700] >>> output = np.kron(np.eye(2), np.ones((2,2))) >>> print(output) [[1. 1. 0. 0.] [1. 1. 0. 0.] [0. 0. 1. 1.] [0. 0. 1. 1.]]
-
tinyms.lcm(x1, x2, dtype=None)[source]¶ Returns the lowest common multiple of
|x1|and|x2|.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the lowest common multiple of the absolute value of the inputs. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.lcm(np.arange(6), np.array(20)) >>> print(output) [ 0 20 20 60 20 20]
-
tinyms.less(x1, x2, dtype=None)[source]¶ Returns the truth value of
(x1 < x2)element-wise.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, element-wise comparison of x1 and x2. Typically of type bool, unless dtype is passed. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.less(np.array([1, 2]), np.array([2, 2])) >>> print(output) [ True False]
-
tinyms.less_equal(x1, x2, dtype=None)[source]¶ Returns the truth value of
(x1 <= x2)element-wise.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, element-wise comparison of x1 and x2. Typically of type bool, unless dtype is passed. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.less_equal(np.array([4, 2, 1]), np.array([2, 2, 2])) >>> print(output) [False True True]
-
tinyms.linspace(start, stop, num=50, endpoint=True, retstep=False, dtype=None, axis=0)[source]¶ Returns evenly spaced values within a given interval.
- Parameters
start (Union[int, list(int), tuple(int), tensor]) – The starting value of the sequence.
stop (Union[int, list(int), tuple(int), tensor]) – The end value of the sequence, unless endpoint is set to False. In that case, the sequence consists of all but the last of num + 1 evenly spaced samples, so that stop is excluded. Note that the step size changes when endpoint is False.
num (int, optional) – Number of samples to generate. Default is 50.
endpoint (bool, optional) – If True, stop is the last sample. Otherwise, it is not included. Default is True.
retstep (bool, optional) – If True, return (samples, step), where step is the spacing between samples.
dtype (Union[
mindspore.dtype, str], optional) – Designated tensor dtype, If dtype is None, infer the data type from other input arguments. Default is None.axis (int, optional) – The axis in the result to store the samples. Relevant only if start or stop are array-like. By default \((0)\), the samples will be along a new axis inserted at the beginning. Use \(-1\) to get an axis at the end. Default is \(0\).
- Returns
Tensor, with num equally spaced samples in the closed interval \([start, stop]\) or the half-open interval \([start, stop)\) (depending on whether endpoint is True or False).
Step, the size of spacing between samples, only returned if retstep is True.
- Raises
TypeError – If input arguments have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.linspace(0, 5, 6)) [0. 1. 2. 3. 4. 5.]
-
tinyms.log(x, dtype=None)[source]¶ Returns the natural logarithm, element-wise.
The natural logarithm log is the inverse of the exponential function, so that
log(exp(x)) = x. The natural logarithm is logarithm in base e.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16, and np.float32. On CPU, the supported dtypes are np.float16, np.float32, and np.float64.
- Parameters
x (Tensor) – Input array. For integer arguments with absolute value larger than 1 the result is always zero because of the way Python handles integer division. For integer zero the result is an overflow.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, the natural logarithm of x, element-wise. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([2, 3, 4]).astype('float32') >>> output = np.log(x) >>> print(output) [0.69314575 1.09861 1.3862929 ]
-
tinyms.log10(x, dtype=None)[source]¶ Base-10 logarithm of x.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([10, 100, 1000]).astype('float16') >>> output = np.log10(x) >>> print(output) [1. 2. 3.]
-
tinyms.log1p(x, dtype=None)[source]¶ Returns the natural logarithm of one plus the input array, element-wise.
Calculates
log(1 + x).Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input array.
dtype (
mindspore.dtype) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([1, 2, 3]).astype('float16') >>> output = np.log1p(x) >>> print(output) [0.6934 1.099 1.387 ]
-
tinyms.log2(x, dtype=None)[source]¶ Base-2 logarithm of x.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([2, 4, 8]).astype('float16') >>> output = np.log2(x) >>> print(output) [1. 2. 3.]
-
tinyms.logaddexp(x1, x2, dtype=None)[source]¶ Logarithm of the sum of exponentiations of the inputs.
Calculates
log(exp(x1) + exp(x2)). This function is useful in statistics where the calculated probabilities of events may be so small as to exceed the range of normal floating point numbers. In such cases the logarithm of the calculated probability is stored. This function allows adding probabilities stored in such a fashion.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([1, 2, 3]).astype('float16') >>> x2 = np.array(2).astype('float16') >>> output = np.logaddexp(x1, x2) >>> print(output) [2.312 2.693 3.312]
-
tinyms.logaddexp2(x1, x2, dtype=None)[source]¶ Logarithm of the sum of exponentiations of the inputs in base of 2.
Calculates
log2(2**x1 + 2**x2). This function is useful in machine learning when the calculated probabilities of events may be so small as to exceed the range of normal floating point numbers. In such cases the base-2 logarithm of the calculated probability can be used instead. This function allows adding probabilities stored in such a fashion.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([2, 4, 8]).astype('float16') >>> x2 = np.array(2).astype('float16') >>> output = np.logaddexp2(x1, x2) >>> print(output) [3. 4.32 8.02]
-
tinyms.logical_and(x1, x2, dtype=None)[source]¶ Computes the truth value of x1 AND x2 element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar. Boolean result of the logical AND operation applied to the elements of x1 and x2; the shape is determined by broadcasting. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([True, False]) >>> x2 = np.array([False, False]) >>> output = np.logical_and(x1, x2) >>> print(output) [False False]
-
tinyms.logical_not(a, dtype=None)[source]¶ Computes the truth value of NOT a element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
a (Tensor) – The input tensor whose dtype is bool.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. Boolean result with the same shape as a of the NOT operation on elements of a. This is a scalar if a is a scalar.
- Raises
TypeError – if the input is not a tensor or its dtype is not bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([True, False]) >>> output = np.logical_not(a) >>> print(output) [False True]
-
tinyms.logical_or(x1, x2, dtype=None)[source]¶ Computes the truth value of x1 OR x2 element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, element-wise comparison of x1 and x2. Typically of type bool, unless
dtype=objectis passed. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([True, False]) >>> x2 = np.array([False, True]) >>> output = np.logical_or(x1, x2) >>> print(output) [ True True]
-
tinyms.logical_xor(x1, x2, dtype=None)[source]¶ Computes the truth value of x1 XOR x2, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar. Boolean result of the logical AND operation applied to the elements of x1 and x2; the shape is determined by broadcasting. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([True, False]) >>> x2 = np.array([False, False]) >>> output = np.logical_xor(x1, x2) >>> print(output) [True False]
-
tinyms.logspace(start, stop, num=50, endpoint=True, base=10.0, dtype=None, axis=0)[source]¶ Returns numbers spaced evenly on a log scale.
In linear space, the sequence starts at base ** start (base to the power of start) and ends with base ** stop (see endpoint below).
- Parameters
start (Union[int, list(int), tuple(int), tensor]) –
base ** startis the starting value of the sequence.stop (Union[int, list(int), tuple(int), tensor]) –
base ** stopis the final value of the sequence, unless endpoint is False. In that case,num + 1values are spaced over the interval in log-space, of which all but the last (a sequence of length num) are returned.num (int, optional) – Number of samples to generate. Default is 50.
endpoint (bool, optional) – If True, stop is the last sample. Otherwise, it is not included. Default is True.
base (Union[int, float], optional) – The base of the log space. The step size between the elements in \(ln(samples) / ln(base)\) (or \(log_{base}(samples)\)) is uniform. Default is \(10.0\).
dtype (Union[
mindspore.dtype, str], optional) – Designated tensor dtype. If dtype is None, infer the data type from other input arguments. Default is None.axis (int, optional) – The axis in the result to store the samples. Relevant only if start or stop is array-like. By default (\(0\)), the samples will be along a new axis inserted at the beginning. Use \(-1\) to get an axis at the end. Default is \(0\).
- Returns
Tensor, equally spaced on a log scale.
- Raises
TypeError – If input arguments have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.logspace(0, 5, 6, base=2.0)) [ 1. 2. 4. 8. 16. 32.]
-
tinyms.matmul(x1, x2, dtype=None)[source]¶ Returns the matrix product of two arrays.
Note
Numpy arguments out, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16 and np.float32. On CPU, the supported dtypes are np.float16 and np.float32.
- Parameters
- Returns
Tensor or scalar, the matrix product of the inputs. This is a scalar only when both x1, x2 are 1-d vectors.
- Raises
ValueError – If the last dimension of x1 is not the same size as the second-to-last dimension of x2, or if a scalar value is passed in.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.arange(2*3*4).reshape(2, 3, 4).astype('float32') >>> x2 = np.arange(4*5).reshape(4, 5).astype('float32') >>> output = np.matmul(x1, x2) >>> print(output) [[[ 70. 76. 82. 88. 94.] [ 190. 212. 234. 256. 278.] [ 310. 348. 386. 424. 462.]] [[ 430. 484. 538. 592. 646.] [ 550. 620. 690. 760. 830.] [ 670. 756. 842. 928. 1014.]]]
-
tinyms.max(a, axis=None, keepdims=False, initial=None, where=True)¶ Returns the maximum of an array or maximum along an axis.
Note
Numpy argument out is not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
a (Tensor) – Input data.
axis (None or int or tuple of ints, optional) – defaults to None. Axis or axes along which to operate. By default, flattened input is used. If this is a tuple of ints, the maximum is selected over multiple axes, instead of a single axis or all the axes as before.
keepdims (boolean, optional) – defaults to False. If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.
initial (scalar, optional) – The minimum value of an output element. Must be present to allow computation on empty slice.
where (boolean Tensor, optional) – defaults to True. A boolean array which is broadcasted to match the dimensions of array, and selects elements to include in the reduction. If non-default value is passed, initial must also be provided.
- Returns
Tensor or scalar, maximum of a. If axis is None, the result is a scalar value. If axis is given, the result is an array of dimension
a.ndim - 1.- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.arange(4).reshape((2,2)).astype('float32') >>> output = np.amax(a) >>> print(output) 3.0 >>> output = np.amax(a, axis=0) >>> print(output) [2. 3.] >>> output = np.amax(a, axis=1) >>> print(output) [1. 3.] >>> output = np.amax(a, where=np.array([False, True]), initial=-1, axis=0) >>> print(output) [-1. 3.]
-
tinyms.maximum(x1, x2, dtype=None)[source]¶ Returns the element-wise maximum of array elements.
Compares two arrays and returns a new array containing the element-wise maxima.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On Ascend, input arrays containing inf or NaN are not supported.
- Parameters
- Returns
Tensor or scalar, the maximum of x1 and x2, element-wise. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.maximum(np.array([2, 3, 4]), np.array([1, 5, 2])) >>> print(output) [2 5 4]
-
tinyms.mean(a, axis=None, keepdims=False, dtype=None)[source]¶ Computes the arithmetic mean along the specified axis.
Returns the average of the array elements. The average is taken over the flattened array by default, otherwise over the specified axis.
Note
Numpy arguments out is not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
a (Tensor) – input tensor containing numbers whose mean is desired. If a is not an array, a conversion is attempted.
axis (None or int or tuple of ints, optional) – Axis or axes along which the means are computed. The default is to compute the mean of the flattened array. If this is a tuple of ints, a mean is performed over multiple axes.
keepdims (bool, optional) – If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input tensor.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, an array containing the mean values.
- Raises
ValueError – if axes are out of the range of
[-a.ndim, a.ndim), or if the axes contain duplicates.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.arange(6, dtype='float32') >>> output = np.mean(a, 0) >>> print(output) 2.5
-
tinyms.meshgrid(*xi, sparse=False, indexing='xy')[source]¶ Returns coordinate matrices from coordinate vectors.
Make N-D coordinate arrays for vectorized evaluations of N-D scalar/vector fields over N-D grids, given one-dimensional coordinate arrays x1, x2,…, xn.
Note
Numpy argument copy is not supported, and a copy is always returned.
- Parameters
*xi (Tensor) – 1-D arrays representing the coordinates of a grid.
indexing (‘xy’, ‘ij’, optional) – Cartesian (‘xy’, default) or matrix (‘ij’) indexing of output. In the 2-D case with inputs of length M and N, the outputs are of shape (N, M) for ‘xy’ indexing and (M, N) for ‘ij’ indexing. In the 3-D case with inputs of length M, N and P, outputs are of shape (N, M, P) for ‘xy’ indexing and (M, N, P) for ‘ij’ indexing.
sparse (bool, optional) – If True a sparse grid is returned in order to conserve memory. Default is False.
- Returns
Tuple of tensors, for vectors x1, x2,…, xn with lengths
Ni=len(xi), return (N1, N2, N3,…Nn) shaped arrays ifindexing=’ij’or (N2, N1, N3,…Nn) shaped arrays ifindexing=’xy’with the elements of xi repeated to fill the matrix along the first dimension for x1, the second for x2 and so on.- Raises
TypeError – if the input is not a tensor, or sparse is not boolean, or indexing is not ‘xy’ or ‘ij’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.linspace(0, 1, 3) >>> y = np.linspace(0, 1, 2) >>> xv, yv = np.meshgrid(x, y) >>> print(xv) [[0. 0.5 1. ] [0. 0.5 1. ]] >>> print(yv) [[0. 0. 0.] [1. 1. 1.]] >>> xv, yv = np.meshgrid(x, y, sparse=True) >>> print(xv) [[0. 0.5 1. ]] >>> print(yv) [[0.] [1.]]
-
tinyms.min(a, axis=None, keepdims=False, initial=None, where=True)¶ Returns the minimum of an array or minimum along an axis.
Note
Numpy argument out is not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
a (Tensor) – Input data.
axis (None or int or tuple of ints, optional) – defaults to None. Axis or axes along which to operate. By default, flattened input is used. If this is a tuple of ints, the minimum is selected over multiple axes, instead of a single axis or all the axes as before.
keepdims (boolean, optional) – defaults to False. If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array.
initial (scalar, optional) – The maximum value of an output element. Must be present to allow computation on empty slice.
where (boolean Tensor, optional) – defaults to True. A boolean array which is broadcasted to match the dimensions of array, and selects elements to include in the reduction. If non-default value is passed, initial must also be provided.
- Returns
Tensor or scalar, minimum of a. If axis is None, the result is a scalar value. If axis is given, the result is an array of dimension
a.ndim - 1.- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.arange(4).reshape((2,2)).astype('float32') >>> output = np.amin(a) >>> print(output) 0.0 >>> output = np.amin(a, axis=0) >>> print(output) [0. 1.] >>> output = np.amin(a, axis=1) >>> print(output) [0. 2.] >>> output = np.amin(a, where=np.array([False, True]), initial=10, axis=0) >>> print(output) [10. 1.]
-
tinyms.minimum(x1, x2, dtype=None)[source]¶ Element-wise minimum of tensor elements.
Compares two tensors and returns a new tensor containing the element-wise minima.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On Ascend, input arrays containing inf or NaN are not supported.
- Parameters
- Returns
Tensor, element-wise minimum of x1 and x2.
- Raises
TypeError – If inputs have types not specified above.
ValueError – If the shapes of x1 and x2 cannot be broadcast.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.asarray([1, 2]) >>> b = np.asarray([[1, 3],[1, 4]]) >>> print(np.minimum(a, b)) [[1 2] [1 2]]
-
tinyms.mod(x1, x2, dtype=None)¶ Returns element-wise remainder of division.
Computes the remainder complementary to the floor_divide function. It is equivalent to the Python modulus operator
x1 % x2and has the same sign as the divisor x2. The MATLAB function equivalent to np.remainder is mod.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the element-wise remainder of the quotient
floor_divide(x1, x2). This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.remainder(np.array([4, 7]), np.array([2, 3])) >>> print(output) [0 1] >>> output = np.remainder(np.arange(7), np.array(5)) >>> print(output) [0 1 2 3 4 0 1]
-
tinyms.moveaxis(a, source, destination)[source]¶ Moves axes of an array to new positions.
Other axes remain in their original order.
- Parameters
- Returns
Tensor, array with moved axes.
- Raises
ValueError – if axes are out of the range of
[-a.ndim, a.ndim), or if the axes contain duplicates.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.zeros((3, 4, 5)) >>> output = np.moveaxis(x, 0, -1) >>> print(output.shape) (4, 5, 3) >>> output = np.moveaxis(x, -1, 0) >>> print(output.shape) (5, 3, 4) >>> output = np.moveaxis(x, [0, 1, 2], [-1, -2, -3]) >>> print(output.shape) (5, 4, 3)
-
tinyms.multiply(x1, x2, dtype=None)[source]¶ Multiplies arguments element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the product of x1 and x2, element-wise. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.full((3, 2), [1, 2]) >>> x2 = np.full((3, 2), [3, 4]) >>> output = np.multiply(x1, x2) >>> print(output) [[3 8] [3 8] [3 8]]
-
tinyms.nancumsum(a, axis=None, dtype=None)[source]¶ Return the cumulative sum of array elements over a given axis treating Not a Numbers (NaNs) as zero. The cumulative sum does not change when NaNs are encountered and leading NaNs are replaced by zeros.
Zeros are returned for slices that are all-NaN or empty.
Note
If
a.dtypeisint8,int16orbool, the result dtype will be elevated toint32.- Parameters
a (Tensor) – Input tensor.
axis (int, optional) – Axis along which the cumulative sum is computed. The default (None) is to compute the cumsum over the flattened array.
dtype (
mindspore.dtype, optional) – If not specified, stay the same as a, unless a has an integer dtype with a precision less than that of the default platform integer. In that case, the default platform integer is used.
- Returns
Tensor.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If axis is out of range.
- Supported Platforms:
GPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([[1, 2], [3, np.nan]]) >>> output = np.nancumsum(a) >>> print(output) [1. 3. 6. 6.] >>> output = np.nancumsum(a, axis=0) >>> print(output) [[1. 2.] [4. 2.]] >>> output = np.nancumsum(a, axis=1) >>> print(output) [[1. 3.] [3. 3.]]
-
tinyms.nanmean(a, axis=None, dtype=None, keepdims=False)[source]¶ Computes the arithmetic mean along the specified axis, ignoring NaNs.
Returns the average of the array elements. The average is taken over the flattened array by default, otherwise over the specified axis. float32 intermediate and return values are used for integer inputs.
Note
Numpy arguments out is not supported.
- Parameters
a (Union[int, float, bool, list, tuple, Tensor]) – Array containing numbers whose mean is desired. If a is not an array, a conversion is attempted.
axis (Union[int, tuple of int, None], optional) – Axis or axes along which the mean is computed. The default is to compute the mean of the flattened array.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.keepdims (boolean, optional) – defaults to False. If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original a.
- Returns
Tensor.
- Raises
ValueError – if axes are out of the range of
[-a.ndim, a.ndim), or if the axes contain duplicates.
- Supported Platforms:
GPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([[1, np.nan], [3, 4]]) >>> output = np.nanmean(a) >>> print(output) 2.6666667 >>> output = np.nanmean(a, axis=0) >>> print(output) [2. 4.] >>> output = np.nanmean(a, axis=1) >>> print(output) [1. 3.5]
-
tinyms.nanstd(a, axis=None, dtype=None, ddof=0, keepdims=False)[source]¶ Computes the standard deviation along the specified axis, while ignoring NaNs.
Returns the standard deviation, a measure of the spread of a distribution, of the non-NaN array elements. The standard deviation is computed for the flattened array by default, otherwise over the specified axis.
Note
Numpy arguments out is not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
a (Union[int, float, bool, list, tuple, Tensor]) – Calculates the standard deviation of the non-NaN values.
axis (Union[int, tuple of int, None], optional) – Axis or axes along which the standard deviation is computed. The default is to compute the standard deviation of the flattened array.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.ddof (int, optional) – “Delta Degrees of Freedom”: the divisor used in the calculation is
N - ddof, where N represents the number of non-NaN elements. By default ddof is zero.keepdims (boolean, optional) – defaults to False. If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original a.
- Returns
Tensor.
- Raises
ValueError – if axes are out of the range of
[-a.ndim, a.ndim), or if the axes contain duplicates.
- Supported Platforms:
GPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([[1, np.nan], [3, 4]]) >>> output = np.nanvar(a) >>> print(output) 1.5555557 >>> output = np.nanvar(a, axis=0) >>> print(output) [1. 0.] >>> output = np.nanvar(a, axis=1) >>> print(output) [0. 0.25]
-
tinyms.nansum(a, axis=None, dtype=None, keepdims=False)[source]¶ Returns the sum of array elements over a given axis treating Not a Numbers (NaNs) as zero.
Note
Numpy arguments out is not supported.
- Parameters
a (Union[int, float, bool, list, tuple, Tensor]) – Array containing numbers whose sum is desired. If a is not an array, a conversion is attempted.
axis (Union[int, tuple of int, None], optional) – Axis or axes along which the sum is computed. The default is to compute the sum of the flattened array.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.keepdims (boolean, optional) – defaults to False. If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original a.
- Returns
Tensor.
- Raises
ValueError – if axes are out of the range of
[-a.ndim, a.ndim), or if the axes contain duplicates.
- Supported Platforms:
GPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([[1, 1], [1, np.nan]]) >>> output = np.nansum(a) >>> print(output) 3.0 >>> output = np.nansum(a, axis=0) >>> print(output) [2. 1.]
-
tinyms.nanvar(a, axis=None, dtype=None, ddof=0, keepdims=False)[source]¶ Computes the variance along the specified axis, while ignoring NaNs.
Returns the variance of the array elements, a measure of the spread of a distribution. The variance is computed for the flattened array by default, otherwise over the specified axis.
Note
Numpy arguments out is not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
a (Union[int, float, bool, list, tuple, Tensor]) – Array containing numbers whose variance is desired. If a is not an array, a conversion is attempted.
axis (Union[int, tuple of int, None], optional) – Axis or axes along which the variance is computed. The default is to compute the variance of the flattened array.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.ddof (int, optional) – “Delta Degrees of Freedom”: the divisor used in the calculation is
N - ddof, where N represents the number of non-NaN elements. By default ddof is zero.keepdims (boolean, optional) – defaults to False. If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the original a.
- Returns
Tensor.
- Raises
ValueError – if axes are out of the range of
[-a.ndim, a.ndim), or if the axes contain duplicates.
- Supported Platforms:
GPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([[1, np.nan], [3, 4]]) >>> output = np.nanstd(a) >>> print(output) 1.2472192 >>> output = np.nanstd(a, axis=0) >>> print(output) [1. 0.] >>> output = np.nanstd(a, axis=1) >>> print(output) [0. 0.5]
-
tinyms.negative(a, dtype=None)[source]¶ Numerical negative, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
a (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.asarray([1, -1]).astype('float32') >>> output = np.negative(a) >>> print(output) [-1. 1.]
-
tinyms.not_equal(x1, x2, dtype=None)[source]¶ Returns (x1 != x2) element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, element-wise comparison of x1 and x2. Typically of type bool, unless dtype is passed. This is a scalar if both x1 and x2 are scalars.
- Raises
TypeError – If the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.asarray([1, 2]) >>> b = np.asarray([[1, 3],[1, 4]]) >>> print(np.not_equal(a, b)) [[False True] [False True]]
-
tinyms.not_equal(x1, x2, dtype=None)[source] Returns (x1 != x2) element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, element-wise comparison of x1 and x2. Typically of type bool, unless dtype is passed. This is a scalar if both x1 and x2 are scalars.
- Raises
TypeError – If the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.asarray([1, 2]) >>> b = np.asarray([[1, 3],[1, 4]]) >>> print(np.not_equal(a, b)) [[False True] [False True]]
-
tinyms.ones(shape, dtype=mindspore.float32)[source]¶ Returns a new tensor of given shape and type, filled with ones.
- Parameters
- Returns
Tensor, with the designated shape and dtype, filled with ones.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If shape entries have values \(< 0\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.ones((2,2))) [[1. 1.] [1. 1.]]
-
tinyms.ones_like(a, dtype=None, shape=None)[source]¶ Returns an array of ones with the same shape and type as a given array.
Note
Input array must have the same size across a dimension. If a is not a Tensor, dtype is float32 by default if not provided.
- Parameters
- Returns
Tensor, array of ones with the same shape and type as a.
- Raises
ValueError – if a is not a Tensor, list or tuple.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((4,1,2)) >>> output = np.ones_like(a) >>> print(output) [[[1. 1.]] [[1. 1.]] [[1. 1.]] [[1. 1.]]]
-
tinyms.outer(a, b)[source]¶ Computes the outer product of two vectors.
Given two vectors,
a = [a0, a1, ..., aM]andb = [b0, b1, ..., bN], the outer product is:[[a0*b0 a0*b1 ... a0*bN ][a1*b0 . ][ ... . ][aM*b0 aM*bN ]]Note
Numpy argument
outis not supported. On GPU, the supported dtypes are np.float16, and np.float32. On CPU, the supported dtypes are np.float16, np.float32, and np.float64.- Parameters
- Returns
Tensor or scalar,
out[i, j] = a[i] * b[j].- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.full(7, 2).astype('float32') >>> b = np.full(4, 3).astype('float32') >>> output = np.outer(a, b) >>> print(output) [[6. 6. 6. 6.] [6. 6. 6. 6.] [6. 6. 6. 6.] [6. 6. 6. 6.] [6. 6. 6. 6.] [6. 6. 6. 6.] [6. 6. 6. 6.]]
-
tinyms.positive(a, dtype=None)[source]¶ Numerical positive, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
a (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.asarray([1, -1]).astype('float32') >>> output = np.positive(a) >>> print(output) [1. -1.]
-
tinyms.power(x1, x2, dtype=None)[source]¶ First array elements raised to powers from second array, element-wise.
Raises each base in x1 to the positionally-corresponding power in x2.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16, and np.float32.
- Parameters
- Returns
Tensor or scalar, the bases in x1 raised to the exponents in x2. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.full((3, 2), [1, 2]).astype('float32') >>> x2 = np.full((3, 2), [3, 4]).astype('float32') >>> output = np.power(x1, x2) >>> print(output) [[ 1. 16.] [ 1. 16.] [ 1. 16.]]
-
tinyms.promote_types(type1, type2)[source]¶ Returns the data type with the smallest size and smallest scalar kind.
Note
The promotion rule is slightly different from original Numpy, but more like jax, due to the preference on
32-bitover64-bitdata types.- Parameters
type1 (Union[
mindspore.dtype, str]) – First data type.type2 (Union[
mindspore.dtype, str]) – Second data type.
- Returns
The promoted data type.
- Raises
TypeError – if the input are not valid
mindspore.dtypeinput.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.promote_types(np.float32, np.float64) >>> print(output) Float64
-
tinyms.ptp(x, axis=None, keepdims=False)[source]¶ Range of values (maximum - minimum) along an axis. The name of the function comes from the acronym for ‘peak to peak’.
Note
Numpy arguments dtype and out are not supported.
- Parameters
- Returns
Tensor.
- Raises
TypeError – if inputs have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([[4.0, 9.0, 2.0, 10.0], [6.0, 9.0, 7.0, 12.0]]) >>> print(np.ptp(x, axis=1)) [8. 6.] >>> print(np.ptp(x, axis=0)) [2. 0. 5. 2.]
-
tinyms.rad2deg(x, dtype=None)[source]¶ Converts angles from radians to degrees.
- Parameters
x (Tensor) – Angles in radians.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor, the corresponding angle in degrees. This is a tensor scalar if x is a tensor scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([1, 2, 3, -4, -5]) >>> output = np.rad2deg(x) >>> print(output) [ 57.295776 114.59155 171.88733 -229.1831 -286.47888 ]
-
tinyms.ravel(x)[source]¶ Returns a contiguous flattened tensor.
A 1-D tensor, containing the elements of the input, is returned.
- Parameters
x (Tensor) – A tensor to be flattened.
- Returns
Flattened tensor, has the same data type as the original tensor x.
- Raises
TypeError – If x is not tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((2,3,4)) >>> output = np.ravel(x) >>> print(output.shape) (24,)
-
tinyms.reciprocal(x, dtype=None)[source]¶ Returns the reciprocal of the argument, element-wise.
Calculates
1/x.Note
Numpy arguments casting, order, subok, signature, and extobj are not supported. When where is provided, out must have a tensor value. out is not supported for storing the result, however it can be used in combination with where to set the value at indices for which where is set to False.
- Parameters
x (Tensor) – Input array. For integer arguments with absolute value larger than 1 the result is always zero because of the way Python handles integer division. For integer zero the result is an overflow.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, this is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(1, 7).reshape(2, 3).astype('float32') >>> output = np.reciprocal(x) >>> print(output) [[1. 0.5 0.33333334] [0.25 0.2 0.16666667]]
-
tinyms.remainder(x1, x2, dtype=None)[source]¶ Returns element-wise remainder of division.
Computes the remainder complementary to the floor_divide function. It is equivalent to the Python modulus operator
x1 % x2and has the same sign as the divisor x2. The MATLAB function equivalent to np.remainder is mod.Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the element-wise remainder of the quotient
floor_divide(x1, x2). This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.remainder(np.array([4, 7]), np.array([2, 3])) >>> print(output) [0 1] >>> output = np.remainder(np.arange(7), np.array(5)) >>> print(output) [0 1 2 3 4 0 1]
-
tinyms.repeat(a, repeats, axis=None)[source]¶ Repeats elements of an array.
- Parameters
a (Tensor) – Input array.
repeats (int or sequence of ints) – The number of repetitions for each element. repeats is broadcasted to fit the shape of the given axis.
axis (int, optional) – The axis along which to repeat values. By default, use the flattened input array, and return a flat output array.
- Returns
Tensor, output array which has the same shape as a, except along the given axis.
- Raises
ValueError – if axis is out of range.
TypeError – if input a is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.repeat(np.array(3), 4) >>> print(output) [3 3 3 3] >>> x = np.array([[1,2],[3,4]]) >>> output = np.repeat(x, 2) >>> print(output) [1 1 2 2 3 3 4 4] >>> output = np.repeat(x, 3, axis=1) >>> print(output) [[1 1 1 2 2 2] [3 3 3 4 4 4]] >>> output = np.repeat(x, [1, 2], axis=0) >>> print(output) [[1 2] [3 4] [3 4]]
-
tinyms.reshape(x, new_shape)[source]¶ Reshapes a tensor without changing its data.
- Parameters
x (Tensor) – A tensor to be reshaped.
new_shape (Union[int, list(int), tuple(int)]) – The new shape should be compatible with the original shape. If the tuple has only one element, the result will be a 1-D tensor of that length. One shape dimension can be \(-1\). In this case, the value is inferred from the length of the tensor and remaining dimensions.
- Returns
Reshaped Tensor. Has the same data type as the original tensor x.
- Raises
TypeError – If new_shape is not integer, list or tuple, or x is not tensor.
ValueError – If new_shape is not compatible with the original shape.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.asarray([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]) >>> output = np.reshape(x, (3, 2)) >>> print(output) [[-0.1 0.3] [ 3.6 0.4] [ 0.5 -3.2]] >>> output = np.reshape(x, (3, -1)) >>> print(output) [[-0.1 0.3] [ 3.6 0.4] [ 0.5 -3.2]] >>> output = np.reshape(x, (6, )) >>> print(output) [-0.1 0.3 3.6 0.4 0.5 -3.2]
-
tinyms.roll(a, shift, axis=None)[source]¶ Rolls a tensor along given axes.
Elements that rolls beyond the last position are re-introduced at the first.
- Parameters
a (Tensor) – Input tensor.
shift (Union[int, tuple(int) – The number of places by which elements are shifted. If a tuple, then axis must be a tuple of the same size, and each of the given axes is shifted by the corresponding number. If shift is an int while axis is a tuple of ints, then the same value is used for all given axes.
axis (Union[int, tuple(int)], optional) – Axis or axes along which elements are shifted. By default, the array is flattened before shifting, after which the original shape is restored.
- Returns
Tensor, with the same shape as a.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If axis exceeds a.ndim, or shift and axis cannot broadcast.
Examples
>>> import mindspore.numpy as np >>> a = np.reshape(np.arange(12), (3, 4)) >>> print(np.roll(a, [2,-3], [0,-1])) [[ 7 4 5 6] [11 8 9 10] [ 3 0 1 2]]
-
tinyms.rollaxis(x, axis, start=0)[source]¶ Rolls the specified axis backwards, until it lies in the given position. The positions of the other axes do not change relative to one another.
- Parameters
x (Tensor) – A Tensor to be transposed.
axis (int) – The axis to be rolled.
start (int) –
Default: 0. If \(start <= axis\), the axis is rolled back until it lies in this position (start). If \(start > axis\): the axis is rolled until it lies before this position (start).
If \(start < 0\), the start will be normalized as shown in the table. (Please refer to the source code.)
- Returns
Transposed Tensor. Has the same data type as the original tensor x.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If axis or start is not integer, or x is not tensor.
ValueError – If axis is not in the range of \([-ndim, ndim-1]\) or start is not in the range of \([-ndim, ndim]\).
Examples
>>> import mindspore.numpy as np >>> x = np.ones((2,3,4)) >>> output = np.rollaxis(x, 0, 2) >>> print(output.shape) (3, 2, 4)
-
tinyms.rot90(a, k=1, axes=(0, 1))[source]¶ Rotates a tensor by 90 degrees in the plane specified by axes. Rotation direction is from the first towards the second axis.
- Parameters
- Returns
Tensor.
- Raises
TypeError – if input a is not a Tensor or the argument k is not integer or the argument axes is not tuple of ints or list of ints.
ValueError – if any axis is out of range or the length of axes is not 2.
- Supported Platforms:
GPU
Examples
>>> import mindspore.numpy as np >>> a = np.arange(24).reshape((2, 3, 4)) >>> output = np.rot90(a) >>> print(output) [[[ 8 9 10 11] [20 21 22 23]] [[ 4 5 6 7] [16 17 18 19]] [[ 0 1 2 3] [12 13 14 15]]] >>> output = np.rot90(a, 3, (1, 2)) >>> print(output) [[[ 8 4 0] [ 9 5 1] [10 6 2] [11 7 3]] [[20 16 12] [21 17 13] [22 18 14] [23 19 15]]]
-
tinyms.select(condlist, choicelist, default=0)[source]¶ Returns an array drawn from elements in choicelist, depending on conditions.
- Parameters
condlist (array_like) – The list of conditions which determine from which array in choicelist the output elements are taken. When multiple conditions are satisfied, the first one encountered in condlist is used.
choicelist (array_like) – The list of arrays from which the output elements are taken. It has to be of the same length as condlist.
default (scalar, optional) – The element inserted in output when all conditions evaluate to False.
- Returns
Tensor, the output at position m is the m-th element of the array in choicelist where the m-th element of the corresponding array in condlist is True.
- Supported Platforms:
AscendGPUCPU
- Raises
ValueError – if
len(condlist) != len(choicelist).
Examples
>>> condlist = [[True, True, True, False, False], [False, False, True, False, True]] >>> choicelist = [[0, 1, 2, 3, 4], [0, 1, 4, 9, 16]] >>> output = np.select(condlist, choicelist) >>> print(output) [ 0 1 2 0 16]
-
tinyms.sin(x, dtype=None)[source]¶ Trigonometric sine, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([-5, -1, 0, 2, 4, 100]).astype('float32') >>> output = np.sin(x) >>> print(output) [ 0.9589243 -0.84147096 0. 0.9092974 -0.7568025 -0.50636566]
-
tinyms.sinh(x, dtype=None)[source]¶ Hyperbolic sine, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(5).astype('float32') >>> print(np.sinh(x)) [ 0. 1.1752012 3.6268604 10.017875 27.289917 ]
-
tinyms.split(x, indices_or_sections, axis=0)[source]¶ Splits a tensor into multiple sub-tensors along the given axis.
- Parameters
x (Tensor) – A Tensor to be divided.
indices_or_sections (Union[int, tuple(int), list(int)]) – If integer, \(N\), the tensor will be divided into \(N\) equal tensors along axis. If tuple(int), list(int) or of sorted integers, the entries indicate where along axis the array is split. For example, \([2, 3]\) would, for \(axis=0\), result in three sub-tensors \(x[:2]\), \(x[2:3]\). If an index exceeds the dimension of the array along axis, an empty sub-array is returned correspondingly.
axis (int) – The axis along which to split. Default: 0.
- Returns
A list of sub-tensors.
- Raises
TypeError – If argument indices_or_sections is not integer, tuple(int) or list(int) or argument axis is not integer.
ValueError – If argument axis is out of range of \([-x.ndim, x.ndim)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> input_x = np.arange(9).astype("float32") >>> output = np.split(input_x, 3) >>> print(output) (Tensor(shape=[3], dtype=Float32, value= [ 0.00000000e+00, 1.00000000e+00, 2.00000000e+00]), Tensor(shape=[3], dtype=Float32, value= [ 3.00000000e+00, 4.00000000e+00, 5.00000000e+00]), Tensor(shape=[3], dtype=Float32, value= [ 6.00000000e+00, 7.00000000e+00, 8.00000000e+00]))
-
tinyms.sqrt(x, dtype=None)[source]¶ Returns the non-negative square-root of an array, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16 and np.float32.
- Parameters
x (Tensor) – The values whose square-roots are required.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, an array of the same shape as x, containing the positive square-root of each element in x. For negative elements, nan is returned. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(6).reshape(2, 3).astype('float32') >>> x_squared = np.square(x) >>> output = np.sqrt(x_squared) >>> print(output) [[ 0. 1. 2.] [ 3. 4. 5.]]
-
tinyms.square(x, dtype=None)[source]¶ Returns the element-wise square of the input.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16 and np.float32.
- Parameters
x (Tensor) – Input data.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, element-wise
x*x, of the same shape and dtype as x. This is a scalar if x is a scalar..
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.square(np.arange(6).reshape(2, 3).astype('float32')) >>> print(x) [[ 0. 1. 4.] [ 9. 16. 25.]]
-
tinyms.squeeze(a, axis=None)[source]¶ Removes single-dimensional entries from the shape of an tensor.
- Parameters
- Returns
Tensor, with all or a subset of the dimensions of length \(1\) removed.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If specified axis has shape entry \(> 1\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((1,2,2,1)) >>> x = np.squeeze(x) >>> print(x.shape) (2, 2)
-
tinyms.stack(arrays, axis=0)[source]¶ Joins a sequence of arrays along a new axis.
The axis parameter specifies the index of the new axis in the dimensions of the result. For example, if
axis=0it will be the first dimension and ifaxis=-1it will be the last dimension.Note
Numpy argument out is not supported.
- Parameters
arrays (sequence of Tensor) – Each array must have the same shape.
axis (int, optional) – The axis in the result array along which the input arrays are stacked.
- Returns
Tensor, The stacked array has one more dimension than the input arrays.
- Raises
ValueError – if input is not Tensor, tuple, or list.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> arrays = [np.ones((3, 4)) for _ in range(10)] >>> output = np.stack(arrays, axis=0) >>> print(output.shape) (10, 3, 4) >>> output = np.stack(arrays, axis=1) >>> print(output.shape) (3, 10, 4) >>> output = np.stack(arrays, axis=2) >>> print(output.shape) (3, 4, 10)
-
tinyms.std(x, axis=None, ddof=0, keepdims=False)[source]¶ Computes the standard deviation along the specified axis. The standard deviation is the square root of the average of the squared deviations from the mean, i.e., \(std = sqrt(mean(abs(x - x.mean())**2))\).
Returns the standard deviation, which is computed for the flattened array by default, otherwise over the specified axis.
Note
Numpy arguments dtype and out are not supported.
- Parameters
x (Tensor) – A Tensor to be calculated.
axis (Union[None, int, tuple(int)]) –
Axis or axes along which the standard deviation is computed. Default: None.
If None, compute the standard deviation of the flattened array.
ddof (int) – Means Delta Degrees of Freedom. The divisor used in calculations is \(N - ddof\), where \(N\) represents the number of elements. Default: 0.
keepdims – Default: False.
- Returns
Standard deviation tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> input_x = np.array([1., 2., 3., 4.]) >>> output = np.std(input_x) >>> print(output) 1.118034
-
tinyms.subtract(x1, x2, dtype=None)[source]¶ Subtracts arguments, element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, the difference of x1 and x2, element-wise. This is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.full((3, 2), [1, 2]) >>> x2 = np.full((3, 2), [3, 4]) >>> output = np.subtract(x1, x2) >>> print(output) [[-2 -2] [-2 -2] [-2 -2]]
-
tinyms.swapaxes(x, axis1, axis2)[source]¶ Interchanges two axes of a tensor.
- Parameters
- Returns
Transposed tensor, has the same data type as the original tensor x.
- Raises
TypeError – If axis1 or axis2 is not integer, or x is not tensor.
ValueError – If axis1 or axis2 is not in the range of \([-ndim, ndim-1]\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((2,3,4)) >>> output = np.swapaxes(x, 0, 2) >>> print(output.shape) (4,3,2)
-
tinyms.take(a, indices, axis=None, mode='clip')[source]¶ Takes elements from an array along an axis.
When axis is not None, this function does the same thing as “fancy” indexing (indexing arrays using arrays); however, it can be easier to use if you need elements along a given axis. A call such as
np.take(arr, indices, axis=3)is equivalent toarr[:,:,:,indices,...].Note
Numpy argument out is not supported.
mode = 'raise'is not supported, and the default mode is ‘clip’ instead.- Parameters
a (Tensor) – Source array with shape (Ni…, M, Nk…).
indices (Tensor) – The indices with shape (Nj…) of the values to extract.
axis (int, optional) – The axis over which to select values. By default, the flattened input array is used.
mode (‘raise’, ‘wrap’, ‘clip’, optional) –
Specifies how out-of-bounds indices will behave.
‘raise’ – raise an error (default);
‘wrap’ – wrap around;
‘clip’ – clip to the range. ‘clip’ mode means that all indices that are too large are replaced by the index that addresses the last element along that axis. Note that this disables indexing with negative numbers.
- Returns
Tensor, the indexed result.
- Raises
ValueError – if axis is out of range.
TypeError – if the input is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([4, 3, 5, 7, 6, 8]) >>> indices = np.array([0, 1, 4]) >>> output = np.take(a, indices) >>> print(output) [4 3 6] >>> indices = np.array([[0, 1], [2, 3]]) >>> output = np.take(a, indices) >>> print(output) [[4 3] [5 7]]
-
tinyms.take_along_axis(arr, indices, axis)[source]¶ Takes values from the input array by matching 1d index and data slices.
This iterates over matching 1d slices oriented along the specified axis in the index and data arrays, and uses the former to look up values in the latter. These slices can be different lengths.
- Parameters
arr (Tensor) – Source array with shape (Ni…, M, Nk…).
indices (Tensor) – Indices with shape (Ni…, J, Nk…) to take along each 1d slice of arr. This must match the dimension of arr, but dimensions Ni and Nj only need to broadcast against arr.
axis (int) – The axis to take 1d slices along. If axis is None, the input array is treated as if it had first been flattened to 1d.
- Returns
Tensor, the indexed result, with shape (Ni…, J, Nk…).
- Raises
ValueError – if input array and indices have different number of dimensions.
TypeError – if the input is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Example
>>> import mindspore.numpy as np >>> x = np.arange(12).reshape(3, 4) >>> indices = np.arange(3).reshape(1, 3) >>> output = np.take_along_axis(x, indices, 1) >>> print(output) [[ 0 1 2] [ 4 5 6] [ 8 9 10]]
-
tinyms.tan(x, dtype=None)[source]¶ Computes tangent element-wise.
Equivalent to \(np.sin(x)/np.cos(x)\) element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Raises
TypeError – If the input is not a tensor or is
tensor.dtypeismindsproe.float64.
- Supported Platforms:
AscendCPU
Examples
>>> import mindspore.numpy as np >>> x = np.array([-5, -1, 0, 2, 4, 100]).astype('float32') >>> print(np.tan(x)) [ 3.380515 -1.5574077 0. -2.1850398 1.1578213 -0.58721393]
-
tinyms.tanh(x, dtype=None)[source]¶ Computes hyperbolic tangent element-wise.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – Input tensor.
dtype (
mindspore.dtype, optional) – Default:None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.arange(5).astype('float32') >>> print(np.tanh(x)) [0. 0.7615942 0.9640276 0.9950548 0.9993293]
-
tinyms.tensordot(a, b, axes=2)[source]¶ Computes tensor dot product along specified axes.
Given two tensors, a and b, and an array_like object containing two array_like objects, (a_axes, b_axes), sum the products of a’s and b’s elements (components) over the axes specified by a_axes and b_axes. The third argument can be a single non-negative integer_like scalar, N; if it is such, then the last N dimensions of a and the first N dimensions of b are summed over. Three common use cases are:
axes = 0: tensor productaxes = 1: tensor dot productaxes = 2: (default) tensor double contraction
When axes is integer_like, the sequence for evaluation will be: first the -Nth axis in a and 0th axis in b, and the -1th axis in a and Nth axis in b last. When there is more than one axis to sum over - and they are not the last (first) axes of a (b) - the argument axes should consist of two sequences of the same length, with the first axis to sum over given first in both sequences, the second axis second, and so forth. The shape of the result consists of the non-contracted axes of the first tensor, followed by the non-contracted axes of the second.
Note
On CPU, the supported dypes are np.float16 and np.float32. On GPU, the supported dypes are np.float16 and np.float32.
- Parameters
a (Tensor) – Tensor to “dot”.
b (Tensor) – Tensor to “dot”.
axes (int or sequence of ints) –
integer_like: If an int N, sum over the last N axes of a and the first N axes of b in order. The sizes of the corresponding axes must match.
sequence of ints: Or, a list of axes to be summed over, first sequence applying to a, second to b. Both elements array_like must be of the same length.
- Returns
Tensor, or list of tensors, the tensor dot product of the input.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((3, 4, 5)) >>> b = np.ones((4, 3, 2)) >>> output = np.tensordot(a, b, axes=([1,0],[0,1])) >>> print(output.shape) (5, 2)
-
tinyms.tile(a, reps)[source]¶ Constructs an array by repeating a the number of times given by reps.
If reps has length d, the result will have dimension of
max(d, a.ndim). Ifa.ndim < d, a is promoted to be d-dimensional by prepending new axes. So a shape (3,) array is promoted to (1, 3) for 2-D replication, or shape (1, 1, 3) for 3-D replication. If this is not the desired behavior, promote a to d-dimensions manually before calling this function. Ifa.ndim > d, reps is promoted toa.ndimby pre-pending 1’s to it. Thus for an a of shape (2, 3, 4, 5), a reps of (2, 2) is treated as (1, 1, 2, 2).- Parameters
- Returns
Tensor, the tiled output array.
- Raises
TypeError – if the input is not a tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.array([0, 1, 2]) >>> output = np.tile(a, 2) >>> print(output) [0 1 2 0 1 2] >>> output = np.tile(a, (2, 2)) >>> print(output) [[0 1 2 0 1 2] [0 1 2 0 1 2]] >>> output = np.tile(a, (2, 1, 2)) >>> print(output) [[[0 1 2 0 1 2]] [[0 1 2 0 1 2]]]
-
tinyms.trace(a, offset=0, axis1=0, axis2=1, dtype=None)[source]¶ Returns the sum along diagonals of the array.
If a is 2-D, the sum along its diagonal with the given offset is returned, i.e., the sum of elements
a[i,i+offset]for all i. If a has more than two dimensions, then the axes specified by axis1 and axis2 are used to determine the 2-D sub-arrays whose traces are returned. The shape of the resulting array is the same as that of a with axis1 and axis2 removed.Note
On GPU, the supported dtypes are np.float16, and np.float32. On CPU, the supported dtypes are np.float16, np.float32, and np.float64.
- Parameters
a (Tensor) – Array from which the diagonals are taken.
offset (int, optional) – Offset of the diagonal from the main diagonal. Can be positive or negative. Defaults to main diagonal.
axis1 (int, optional) – Axis to be used as the first axis of the 2-D sub-arrays from which the diagonals should be taken. Defaults to first axis (0).
axis2 (int, optional) – Axis to be used as the second axis of the 2-D sub-arrays from which the diagonals should be taken. Defaults to second axis.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor, sum_along_diagonals. If a is 2-D, the sum along the diagonal is returned. If a has larger dimensions, then an array of sums along diagonals is returned.
- Raises
ValueError – if the input tensor has less than two dimensions.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.trace(np.eye(3)) >>> print(output) 3.0 >>> a = np.arange(8).reshape((2,2,2)) >>> output = np.trace(a) >>> print(output) [6 8] >>> a = np.arange(24).reshape((2,2,2,3)) >>> output = np.trace(a).shape >>> print(output) (2, 3)
-
tinyms.transpose(a, axes=None)[source]¶ Reverses or permutes the axes of a tensor; returns the modified tensor.
- Parameters
- Returns
Tensor, the transposed tensor array.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If the number of axes is not euqal to a.ndim.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x = np.ones((1,2,3)) >>> x = np.transpose(x) >>> print(x.shape) (3, 2, 1)
-
tinyms.trapz(y, x=None, dx=1.0, axis=-1)[source]¶ Integrates along the given axis using the composite trapezoidal rule.
Integrates y (x) along given axis.
- Parameters
y (Tensor) – Input array to integrate.
x (Union[int, float, bool, list, tuple, Tensor], optional) – The sample points corresponding to the y values. If x is None, the sample points are assumed to be evenly spaced dx apart. The default is None.
dx (scalar, optional) – The spacing between sample points when x is None. The default is 1.
axis (int, optional) – The axis along which to integrate.
- Returns
Tensor of float, definite integral as approximated by trapezoidal rule.
- Raises
ValueError – If axis is out of range of
[-y.ndim, y.ndim).
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.arange(6).reshape(2, 3) >>> output = np.trapz(a, x=[-2, 1, 2], axis=1) >>> print(output) [ 3. 15.] >>> output = np.trapz(a, dx=3, axis=0) >>> print(output) [ 4.5 7.5 10.5]
-
tinyms.tri(N, M=None, k=0, dtype=mindspore.float32)[source]¶ Returns a tensor with ones at and below the given diagonal and zeros elsewhere.
- Parameters
N (int) – Number of rows in the array.
M (int, optional) – Number of columns in the array. By default, M is taken equal to N.
k (int, optional) – The sub-diagonal at and below which the array is filled. \(k = 0\) is the main diagonal, while \(k < 0\) is below it, and \(k > 0\) is above. The default is 0.
dtype (
mindspore.dtype, optional) – Data type of the returned array. The default ismindspore.dtype.
- Returns
Tensor with shape (N, M), with its lower triangle filled with ones and zeros elsewhere; in other words \(T[i,j] = 1\) for \(j <= i + k\), \(0\) otherwise.
- Raises
TypeError – If input arguments have types not specified above.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.tri(3, 3, 1) >>> print(output) [[1. 1. 0.] [1. 1. 1.] [1. 1. 1.]]
-
tinyms.tril(m, k=0)[source]¶ Returns a lower triangle of a tensor.
Returns a copy of a tensor with elements above the k-th diagonal zeroed.
- Parameters
- Returns
Lower triangle of m, of same shape and data-type as m.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If input m’s rank \(< 1\).
Examples
>>> import mindspore.numpy as np >>> output = np.tril(np.ones((3, 3))) >>> print(output) [[1. 0. 0.] [1. 1. 0.] [1. 1. 1.]]
-
tinyms.triu(m, k=0)[source]¶ Returns an upper triangle of a tensor.
Returns a copy of a tensor with elements below the k-th diagonal zeroed.
- Parameters
- Returns
Upper triangle of m, of same shape and data-type as m.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If input m’s rank < 1.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.triu(np.ones((3, 3))) >>> print(output) [[1. 1. 1.] [0. 1. 1.] [0. 0. 1.]]
-
tinyms.true_divide(x1, x2, dtype=None)[source]¶ Returns a true division of the inputs, element-wise.
Instead of the Python traditional ‘floor division’, this returns a true division.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
- Returns
Tensor or scalar, this is a scalar if both x1 and x2 are scalars.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> x1 = np.full((3, 2), [1, 2]) >>> x2 = np.full((3, 2), [3, 4]) >>> output = np.true_divide(x1, x2) >>> print(output) [[0.33333334 0.5 ] [0.33333334 0.5 ] [0.33333334 0.5 ]]
-
tinyms.trunc(x, dtype=None)[source]¶ Returns the truncated value of the input, element-wise.
The truncated value of the scalar x is the nearest integer i which is closer to zero than x is. In short, the fractional part of the signed number x is discarded.
Note
Numpy arguments out, where, casting, order, subok, signature, and extobj are not supported.
- Parameters
x (Tensor) – input data.
dtype (
mindspore.dtype, optional) – defaults to None. Overrides the dtype of the output Tensor.
- Returns
Tensor or scalar, the truncated value of each element in x. This is a scalar if x is a scalar.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> output = np.trunc(np.array([-1.7, -1.5, -0.2, 0.2, 1.5, 1.7, 2.0])) >>> print(output) [-1. -1. -0. 0. 1. 1. 2.]
-
tinyms.unique(x, return_inverse=False)[source]¶ Finds the unique elements of a tensor. The input tensor will be flattened first when it has more than one dimension.
Note
Numpy arguments axis, return_index and return_counts are not supported. On CPU, this operator must be executed in graph mode.
- Parameters
- Returns
Tensor or tuple of Tensors. - If return_inverse is False, just return the unique tensor. - If return_inverse is True, return tuple of tensors.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If x is not tensor.
Examples
>>> import mindspore.numpy as np >>> from mindspore import context >>> context.set_context(mode=context.GRAPH_MODE) >>> input_x = np.asarray([1, 2, 2, 2, 3, 4, 5]).astype('int32') >>> output_x = np.unique(input_x) >>> print(output_x) [1 2 3 4 5] >>> output_x = np.unique(input_x, return_inverse=True) >>> print(output_x) (Tensor(shape=[5], dtype=Int32, value= [ 1, 2, 3, 4, 5]), Tensor(shape=[7], dtype=Int32, value= [0, 1, 1, 1, 2, 3, 4]))
-
tinyms.vander(x, N=None, increasing=False)[source]¶ Generates a Vandermonde matrix.
The columns of the output matrix are powers of the input vector. The order of the powers is determined by the increasing boolean argument. Specifically, when increasing is False, the i-th output column is the input vector raised element-wise to the power of \(N - i - 1\). Such a matrix with a geometric progression in each row is named for Alexandre-Theophile Vandermonde.
- Parameters
- Returns
Vandermonde matrix. If increasing is False, the first column is \(x^{(N-1)}\), the second \(x^{(N-2)}\) and so forth. If increasing is True, the columns are \(x^0, x^1, ..., x^{(N-1)}\).
- Raises
TypeError – If inputs have types not specified above.
ValueError – If x is not 1-D, or N < 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.vander([1,2,3,4,5])) [[ 1 1 1 1 1] [ 16 8 4 2 1] [ 81 27 9 3 1] [256 64 16 4 1] [625 125 25 5 1]]
-
tinyms.var(x, axis=None, ddof=0, keepdims=False)[source]¶ Computes the variance along the specified axis. The variance is the average of the squared deviations from the mean, i.e., \(var = mean(abs(x - x.mean())**2)\).
Returns the variance, which is computed for the flattened array by default, otherwise over the specified axis.
Note
Numpy arguments dtype and out are not supported.
- Parameters
x (Tensor) – A Tensor to be calculated.
axis (Union[None, int, tuple(int)]) – Axis or axes along which the variance is computed. The default is to compute the variance of the flattened array. Default: None.
ddof (int) – Means Delta Degrees of Freedom. Default: 0. The divisor used in calculations is \(N - ddof\), where \(N\) represents the number of elements.
keepdims (bool) – Default: False.
- Supported Platforms:
AscendGPUCPU
- Returns
Standard deviation tensor.
Examples
>>> import mindspore.numpy as np >>> input_x = np.array([1., 2., 3., 4.]) >>> output = np.var(input_x) >>> print(output) 1.25
-
tinyms.vsplit(x, indices_or_sections)[source]¶ Splits a tensor into multiple sub-tensors vertically (row-wise). It is equivalent to split with \(axis=0\) (default), the array is always split along the first axis regardless of the array dimension.
- Parameters
x (Tensor) – A Tensor to be divided.
indices_or_sections (Union[int, tuple(int), list(int)]) – If integer, \(N\), the tensor will be divided into \(N\) equal tensors along axis. If tuple(int), list(int) or of sorted integers, the entries indicate where along axis the array is split. For example, \([2, 3]\) would, for \(axis=0\), result in three sub-tensors \(x[:2]\), \(x[2:3]\). If an index exceeds the dimension of the array along axis, an empty sub-array is returned correspondingly.
- Returns
A list of sub-tensors.
- Raises
TypeError – If argument indices_or_sections is not integer.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> input_x = np.arange(9).reshape((3, 3)).astype('float32') >>> output = np.vsplit(input_x, 3) >>> print(output) (Tensor(shape=[1, 3], dtype=Float32, value=[[ 0.00000000e+00, 1.00000000e+00, 2.00000000e+00]]), Tensor(shape=[1, 3], dtype=Float32, value=[[ 3.00000000e+00, 4.00000000e+00, 5.00000000e+00]]), Tensor(shape=[1, 3], dtype=Float32, value=[[ 6.00000000e+00, 7.00000000e+00, 8.00000000e+00]]))
-
tinyms.vstack(tup)[source]¶ Stacks tensors in sequence vertically. This is equivalent to concatenation along the first axis. 1-D tensors should firstly be reshaped to (1, N), and then be concatenated along the first axis.
- Parameters
tup (Union[Tensor, tuple, list]) – A sequence of 1-D or 2-D tensors. The tensors must have the same shape along all but the first axis. 1-D tensors must have the same shape.
- Returns
Stacked Tensor, formed by stacking the given tensors.
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If tup is not Tensor, list or tuple.
ValueError – If tup is empty.
Examples
>>> import mindspore.numpy as np >>> x1 = np.array([1, 2, 3]).astype('int32') >>> x2 = np.array([4, 5, 6]).astype('int32') >>> output = np.vstack((x1, x2)) >>> print(output) [[1 2 3] [4 5 6]]
-
tinyms.where(condition, x=None, y=None)[source]¶ Returns elements chosen from x or y depending on condition.
Note
As nonzero is not supported, neither x or y can be None.
- Parameters
- Returns
Tensor or scalar, with elements from x where condition is True, and elements from y elsewhere.
- Raises
ValueError – if operands cannot be broadcast.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> condition = np.full((1, 1, 2), [False, True]) >>> x = np.full((1, 3, 2), 5) >>> y = np.full((2, 1, 1), 7) >>> output = np.where(condition, x, y) >>> print(output) [[[7 5] [7 5] [7 5]] [[7 5] [7 5] [7 5]]]
-
tinyms.zeros(shape, dtype=mindspore.float32)[source]¶ Returns a new tensor of given shape and type, filled with zeros.
- Parameters
- Returns
Tensor, with the designated shape and dtype, filled with zeros.
- Raises
TypeError – If input arguments have types not specified above.
ValueError – If shape entries have values \(< 0\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> print(np.zeros((2,2))) [[0. 0.] [0. 0.]]
-
tinyms.zeros_like(a, dtype=None, shape=None)[source]¶ Returns an array of zeros with the same shape and type as a given array.
Note
Input array must have the same size across a dimension. If a is not a Tensor, dtype is float32 by default if not provided.
- Parameters
- Returns
Tensor, array of zeros with the same shape and type as a.
- Raises
ValueError – if a is not a Tensor, list or tuple.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.numpy as np >>> a = np.ones((4,1,2)) >>> output = np.zeros_like(a) >>> print(output) [[[0. 0.]] [[0. 0.]] [[0. 0.]] [[0. 0.]]]
tinyms.context¶
The context of TinyMS, used to configure the current execution environment, includeing the execution mode, execution backend and other feature switches.
-
tinyms.context.set_context(**kwargs)[source]¶ Set context for running environment.
Context should be configured before running your program. If there is no configuration, the “Ascend” device target will be used by default. GRAPH_MODE or PYNATIVE_MODE can be set by mode attribute and both modes support all backends, default mode is PYNATIVE_MODE.
When the save_graphs attribute is set to True, attribute of save_graphs_path is used to set the intermediate compilation graph storage path. By default, the graphs are saved in the current directory. For other configurations and arguments, please refer to the corresponding module description, the configuration is optional and can be enabled when needed.
Note
Attribute name is required for setting attributes. The mode is not recommended to be changed after net was initialized because the implementations of some operations are different in graph mode and pynative mode. Default: PYNATIVE_MODE.
Some configurations are device specific, see the below table for details:
Common(CPU/GPU/Ascend)
Ascend
GPU
check_bprop
print_file_path
max_device_memory
device_id
enable_dump
enable_graph_kernel
device_target
save_dump_path
enable_sparse
enable_graph_kernel
max_call_depth
enable_reduce_precision
mode
enable_profiling
reserve_class_name_in_scope
profiling_options
save_graphs
variable_memory_max_size
save_graphs_path
auto_tune_mode
env_config_path
grad_for_scalar
- Parameters
mode (int) – Running in GRAPH_MODE(0) or PYNATIVE_MODE(1). Default: PYNATIVE_MODE(1).
device_target (str) – The target device to run, support “Ascend”, “GPU”, and “CPU”. Default: “Ascend”.
device_id (int) – ID of the target device, the value must be in [0, device_num_per_host-1], while device_num_per_host should be no more than 4096. Default: 0.
save_graphs (bool) – Whether to save graphs. Default: False.
save_graphs_path (str) –
Path to save graphs. Default: “.”.
If the program is executed in the parallel mode, save_graphs_path should consist of the path and the current device id, to ensure that writing file conflicts won’t happen when the different processes try to create the files in the same directory. For example, the device_id can be generated by device_id = os.getenv(“DEVICE_ID”) and the save_graphs_path can be set by context.set_context(save_graphs_path=”path/to/ir/files”+device_id).
enable_graph_kernel (bool) – Whether to enable composition of basic primitives. These primitives would be compiled into a fused kernel automatically. Default: False.
reserve_class_name_in_scope (bool) – Whether to save the network class name in the scope. Default: True.
enable_reduce_precision (bool) – Whether to enable precision reduction. Default: True.
enable_dump (bool) – Whether to enable dump. Default: False.
save_dump_path (str) – When the program is executed on Ascend, operators can dump data in this path. The root dump path is configured in /home/HwHiAiUser/ide_daemon/ide_daemon.cfg. So the real dump path is “{configured root dump path}/{save_dump_path}”. Default: “.”.
variable_memory_max_size (str) – Set the maximum size of the variable memory max size. Default: “0GB”.
enable_profiling (bool) – Whether to open profiling. Default: False.
profiling_options (str) –
Set profiling collection options, operators can profiling data here. The values of profiling collection options are as follows, supporting the collection of multiple data.
output: the saving the path of the profiling collection result file. The directory spectified by this parameter needs to be created in advance on the training environment (container or host side) and ensure that the running user configured during installation has read and write permissions.It supports the configuration of absolute or relative paths(relative to the current path when executing the command line). The absolute path configuration starts with ‘/’, for example:/home/data/output. The relative path configuration directly starts with the directory name,for example:output.
training_trace: collect iterative trajectory data, that is, the training task and software information of the AI software stack, to achieve performance analysis of the training task, focusing on data enhancement, forward and backward calculation, gradient aggregation update and other related data. The value is on/off.
task_trace: collect task trajectory data, that is, the hardware information of the HWTS/AICore of the Ascend 910 processor, and analyze the information of beginning and ending of the task. The value is on/off.
aicpu: collect profiling data enhanced by aicpu data. The value is on/off.
fp_point: specify the start position of the forward operator of the training network iteration trajectory, which is used to record the start timestamp of the forward calculation.The configuration value is the name of the first operator specified in the forward direction. when the value is empty,the system will automatically obtain the forward operator name.
bp_point: specify the end position of the iteration trajectory reversal operator of the training network, record the end timestamp of the backward calculation. The configuration value is the name of the operator after the specified reverse. when the value is empty,the system will automatically obtain the backward operator name.
aic_metrics: the values are as follows: ArithmeticUtilization: percentage statistics of various calculation indicators. PipeUtilization: the time-consuming ratio of calculation unit and handling unit,this item is the default value. Memory: percentage of external memory read and write instructions. MemoryL0: percentage of internal memory read and write instructions. ResourceConflictRatio: proportion of pipline queue instructions.
The profiling_options is like ‘{“output”:’/home/data/output’,’training_trace’:’on’}’
check_bprop (bool) – Whether to check bprop. Default: False.
max_device_memory (str) – Sets the maximum memory available for devices. Currently, it is only supported on GPU. The format is “xxGB”. Default: “1024GB”.
print_file_path (str) – The path of saving print data. If this parameter is set, print data is saved to a file by default, and turns off printing to the screen. If the file already exists, add a timestamp suffix to the file. Default: ‘’.
enable_sparse (bool) – Whether to enable sparsity feature. Default: False.
max_call_depth (int) – Specify the maximum depth of function call. Default: 1000.
env_config_path (str) – Config path for DFX.
auto_tune_mode (str) – The mode of auto tune when op building, get the best tiling performance, default: NO_TUNE. The value must be in [‘RL’, ‘GA’, ‘RL,GA’]. RL: rl_tune; GA: ga_tune; RL,GA: rl_tune/ga_tune(Automatic selection). - rl_tune: Reinforecement Learning tune. - ga_tune: Genetic Algorithm tune.
grad_for_scalar (bool) – Whether to get gradient for scalar. Default: False.
- Raises
ValueError – If input key is not an attribute in context.
Examples
>>> context.set_context(mode=context.GRAPH_MODE) >>> context.set_context(mode=context.PYNATIVE_MODE) >>> context.set_context(device_target="Ascend") >>> context.set_context(device_id=0) >>> context.set_context(save_graphs=True, save_graphs_path="./model.ms") >>> context.set_context(enable_reduce_precision=True) >>> context.set_context(enable_dump=True, save_dump_path=".") >>> context.set_context(reserve_class_name_in_scope=True) >>> context.set_context(variable_memory_max_size="6GB") >>> context.set_context(mode=context.GRAPH_MODE, ... device_target="Ascend",device_id=0, save_graphs=True, ... save_graphs_path="/mindspore") >>> context.set_context(enable_profiling=True, ... profiling_options='{"output":"/home/data/output","training_trace":"on"}') >>> context.set_context(max_device_memory="3.5GB") >>> context.set_context(print_file_path="print.pb") >>> context.set_context(max_call_depth=80) >>> context.set_context(env_config_path="./env_config.json")
-
tinyms.context.get_context(attr_key)[source]¶ Get context attribute value according to the input key.
- Parameters
attr_key (str) – The key of the attribute.
- Returns
Object, The value of given attribute key.
- Raises
ValueError – If input key is not an attribute in context.
-
tinyms.context.set_auto_parallel_context(**kwargs)[source]¶ Set auto parallel context, which is valid only for Ascend and GPU target.
Auto parallel context should be configured before the initialization of your network.
Note
Attribute name is required for setting attributes. If a program has tasks with different parallel modes, then before setting new parallel mode for the next task, interface mindspore.context.reset_auto_parallel_context() needs to be called to reset the configuration. Setting or changing parallel modes must be called before any creating Initializer, otherwise, RuntimeError may be raised when compiling the network.
Some configurations are parallel mode specific, see the below table for details:
Common
AUTO_PARALLEL
device_num
gradient_fp32_sync
global_rank
loss_repeated_mean
gradients_mean
auto_parallel_search_mode
parallel_mode
strategy_ckpt_load_file
all_reduce_fusion_config
strategy_ckpt_save_file
enable_parallel_optimizer
full_batch
pipeline_stages
- Parameters
device_num (int) – Available device number, the value must be in [1, 4096]. Default: 1.
global_rank (int) – Global rank id, the value must be in [0, 4095]. Default: 0.
gradients_mean (bool) – Whether to perform mean operator after allreduce of gradients. “stand_alone” do not support gradients_mean. Default: False.
gradient_fp32_sync (bool) – Run allreduce of gradients in fp32. “stand_alone”, “data_parallel” and “hybrid_parallel” do not support gradient_fp32_sync. Default: True.
parallel_mode (str) –
There are five kinds of parallel modes, “stand_alone”, “data_parallel”, “hybrid_parallel”, “semi_auto_parallel” and “auto_parallel”. Default: “stand_alone”.
stand_alone: Only one processor is working.
data_parallel: Distributes the data across different processors.
hybrid_parallel: Achieves data parallelism and model parallelism manually.
semi_auto_parallel: Achieves data parallelism and model parallelism by setting parallel strategies.
auto_parallel: Achieving parallelism automatically.
auto_parallel_search_mode (str) –
There are two kinds of shard strategy search modes, “recursive_programming” and “dynamic_programming”. Default: “dynamic_programming”.
recursive_programming: Recursive programming search mode.
dynamic_programming: Dynamic programming search mode.
parameter_broadcast (bool) – Whether to broadcast parameters before training. Before training, in order to have the same network initialization parameter values for all devices, broadcast the parameters on device 0 to other devices. Parameter broadcasting in different parallel modes is different, data_parallel mode, all parameters are broadcast except for the parameter whose attribute layerwise_parallel is True. Hybrid_parallel, semi_auto_parallel and auto_parallel mode, the segmented parameters do not participate in broadcasting. Default: False.
strategy_ckpt_load_file (str) – The path to load parallel strategy checkpoint. Default: ‘’
strategy_ckpt_save_file (str) – The path to save parallel strategy checkpoint. Default: ‘’
full_batch (bool) – If you load whole batch datasets in auto_parallel mode, this parameter should be set with True. Default: False.
enable_parallel_optimizer (bool) – This is a developing feature, which shards the weight update computation for data parallel training in the benefit of time and memory saving. Currently, auto and semi auto parallel mode support all optimizers in both Ascend and GPU. Data parallel mode only supports Lamb and AdamWeightDecay in Ascend . Default: False.
all_reduce_fusion_config (list) – Set allreduce fusion strategy by parameters indices. Only support ReduceOp.SUM and HCCL_WORLD_GROUP/NCCL_WORLD_GROUP. No Default, if it is not set, the fusion is closed.
pipeline_stages (int) – Set the stage information for pipeline parallel. This indicates how the devices are distributed alone the pipeline. The total devices will be divided into ‘pipeline_stags’ stages. This currently could only be used when parallel mode semi_auto_parallel is enabled. Default: 1.
- Raises
ValueError – If input key is not attribute in auto parallel context.
Examples
>>> context.set_auto_parallel_context(device_num=8) >>> context.set_auto_parallel_context(global_rank=0) >>> context.set_auto_parallel_context(gradients_mean=True) >>> context.set_auto_parallel_context(gradient_fp32_sync=False) >>> context.set_auto_parallel_context(parallel_mode="auto_parallel") >>> context.set_auto_parallel_context(auto_parallel_search_mode="dynamic_programming") >>> context.set_auto_parallel_context(parameter_broadcast=False) >>> context.set_auto_parallel_context(strategy_ckpt_load_file="./strategy_stage1.ckpt") >>> context.set_auto_parallel_context(strategy_ckpt_save_file="./strategy_stage1.ckpt") >>> context.set_auto_parallel_context(full_batch=True) >>> context.set_auto_parallel_context(enable_parallel_optimizer=False) >>> context.set_auto_parallel_context(all_reduce_fusion_config=[8, 160]) >>> context.set_auto_parallel_context(pipeline_stages=2)
-
tinyms.context.get_auto_parallel_context(attr_key)[source]¶ Get auto parallel context attribute value according to the key.
- Parameters
attr_key (str) – The key of the attribute.
- Returns
Returns attribute value according to the key.
- Raises
ValueError – If input key is not attribute in auto parallel context.
-
tinyms.context.reset_auto_parallel_context()[source]¶ Reset auto parallel context attributes to the default values:
device_num: 1.
global_rank: 0.
gradients_mean: False.
gradient_fp32_sync: True.
parallel_mode: ‘stand_alone’.
auto_parallel_search_mode: ‘dynamic_programming’.
parameter_broadcast: False.
strategy_ckpt_load_file: ‘’.
strategy_ckpt_save_file: ‘’.
full_batch: False.
enable_parallel_optimizer: False.
pipeline_stages: 1.
-
class
tinyms.context.ParallelMode[source]¶ Parallel mode options.
There are five kinds of parallel modes, “STAND_ALONE”, “DATA_PARALLEL”, “HYBRID_PARALLEL”, “SEMI_AUTO_PARALLEL” and “AUTO_PARALLEL”. Default: “STAND_ALONE”.
STAND_ALONE: Only one processor is working.
DATA_PARALLEL: Distributes the data across different processors.
HYBRID_PARALLEL: Achieves data parallelism and model parallelism manually.
SEMI_AUTO_PARALLEL: Achieves data parallelism and model parallelism by setting parallel strategies.
AUTO_PARALLEL: Achieves parallelism automatically.
MODE_LIST: The list of all supported parallel modes.
-
tinyms.context.set_ps_context(**kwargs)[source]¶ Set parameter server training mode context.
Note
Some other environment variables should also be set for parameter server training mode. These environment variables are listed below:
MS_SERVER_NUM # Server number MS_WORKER_NUM # Worker number MS_SCHED_HOST # Scheduler IP address MS_SCHED_PORT # Scheduler port MS_ROLE # The role of this process: MS_SCHED #represents the scheduler, MS_WORKER #represents the worker, MS_PSERVER #represents the Server
- Parameters
enable_ps (bool) – Whether to enable parameter server training mode. Only after enable_ps is set True, the environment variables will be effective. Default: False.
- Raises
ValueError – If input key is not the attribute in parameter server training mode context.
Examples
>>> context.set_ps_context(enable_ps=True)
-
tinyms.context.get_ps_context(attr_key)[source]¶ Get parameter server training mode context attribute value according to the key.
- Parameters
attr_key (str) – The key of the attribute.
- Returns
Returns attribute value according to the key.
- Raises
ValueError – If input key is not attribute in auto parallel context.
tinyms.data¶
-
class
tinyms.data.UnalignedDataset(dataset_path, phase, max_dataset_size=inf, shuffle=True)[source]¶ This dataset class can load unaligned/unpaired datasets.
- Parameters
dataset_path (str) – The path of images (should have subfolders trainA, trainB, testA, testB, etc).
phase (str) – Train or test. It requires two directories in dataset_path, like trainA and trainB to. host training images from domain A ‘{dataset_path}/trainA’ and from domain B ‘{dataset_path}/trainB’ respectively.
max_dataset_size (int) – Maximum number of return image paths.
- Returns
Two domain image path list.
-
class
tinyms.data.GanImageFolderDataset(dataset_path, max_dataset_size=inf)[source]¶ This dataset class can load images from image folder.
-
class
tinyms.data.ImdbDataset(imdb_path, glove_path, embed_size=300)[source]¶ parse aclImdb data to features and labels. sentence->tokenized->encoded->padding->features
- Parameters
Examples
>>> from tinyms.data import ImdbDataset >>> >>> imdb_ds = ImdbDataset('./aclImdb', './glove')
-
class
tinyms.data.BertDataset(data_dir, schema_dir=None, shuffle=True, num_parallel_workers=None)[source]¶ This dataset class can load bert from data folder.
- Parameters
data_dir (str) – ‘{data_dir}/result1.tfrecord’, ‘{data_dir}/result2.tfrecord’, etc.
num_parallel_workers (int) – The number of concurrent workers. Default: None.
shuffle (Union[bool, Shuffle level], optional) –
Perform reshuffling of the data every epoch (default=Shuffle.GLOBAL). If shuffle is False, no shuffling will be performed; If shuffle is True, the behavior is the same as setting shuffle to be Shuffle.GLOBAL Otherwise, there are two levels of shuffling:
Shuffle.GLOBAL: Shuffle both the files and samples.
Shuffle.FILES: Shuffle files only.
schema (Union[str, Schema], optional) – Path to the JSON schema file or schema object (default=None). If the schema is not provided, the meta data from the TFData file is considered the schema.
Examples
>>> from tinyms.data import BertDataset >>> >>> bert_ds = BertDataset('data')
-
class
tinyms.data.DistributedSampler(dataset_size, num_replicas=None, rank=None, shuffle=True)[source]¶ Distributed sampler.
-
class
tinyms.data.CelebADataset(dataset_dir, num_parallel_workers=None, shuffle=None, usage='all', sampler=None, decode=False, extensions=None, num_samples=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing CelebA dataset. Only support to read list_attr_celeba.txt currently, which is the attribute annotations of the dataset.
Note
The generated dataset has two columns [‘image’, ‘attr’]. The image tensor is of the uint8 type. The attribute tensor is of the uint32 type and one hot encoded.
Citation of CelebA dataset.
@article{DBLP:journals/corr/LiuLWT14, author = {Ziwei Liu and Ping Luo and Xiaogang Wang and Xiaoou Tang}, title = {Deep Learning Face Attributes in the Wild}, journal = {CoRR}, volume = {abs/1411.7766}, year = {2014}, url = {http://arxiv.org/abs/1411.7766}, archivePrefix = {arXiv}, eprint = {1411.7766}, timestamp = {Tue, 10 Dec 2019 15:37:26 +0100}, biburl = {https://dblp.org/rec/journals/corr/LiuLWT14.bib}, bibsource = {dblp computer science bibliography, https://dblp.org}, howpublished = {http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html}, description = {CelebFaces Attributes Dataset (CelebA) is a large-scale face attributes dataset with more than 200K celebrity images, each with 40 attribute annotations. The images in this dataset cover large pose variations and background clutter. CelebA has large diversities, large quantities, and rich annotations, including * 10,177 number of identities, * 202,599 number of face images, and * 5 landmark locations, 40 binary attributes annotations per image. The dataset can be employed as the training and test sets for the following computer vision tasks: face attribute recognition, face detection, landmark (or facial part) localization, and face editing & synthesis.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, will use value set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None).
usage (str) – one of ‘all’, ‘train’, ‘valid’ or ‘test’ (default=’all’, will read all samples).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None).
decode (bool, optional) – decode the images after reading (default=False).
extensions (list[str], optional) – List of file extensions to be included in the dataset (default=None).
num_samples (int, optional) – The number of images to be included in the dataset (default=None, will include all images).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, num_samples reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
Examples
>>> celeba_dataset_dir = "/path/to/celeba_dataset_directory" >>> dataset = ds.CelebADataset(dataset_dir=celeba_dataset_dir, usage='train')
-
class
tinyms.data.Cifar100Dataset(dataset_dir, usage=None, num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing Cifar100 dataset.
The generated dataset has three columns [‘image’, ‘coarse_label’, ‘fine_label’]. The type of the image tensor is uint8. The coarse and fine labels are each a scalar uint32 tensor. This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of Cifar100 dataset.
@techreport{Krizhevsky09, author = {Alex Krizhevsky}, title = {Learning multiple layers of features from tiny images}, institution = {}, year = {2009}, howpublished = {http://www.cs.toronto.edu/~kriz/cifar.html}, description = {This dataset is just like the CIFAR-10, except it has 100 classes containing 600 images each. There are 500 training images and 100 testing images per class. The 100 classes in the CIFAR-100 are grouped into 20 superclasses. Each image comes with a "fine" label (the class to which it belongs) and a "coarse" label (the superclass to which it belongs).} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
usage (str, optional) – Usage of this dataset, can be “train”, “test” or “all” . “train” will read from 50,000 train samples, “test” will read from 10,000 test samples, “all” will read from all 60,000 samples. (default=None, all samples)
num_samples (int, optional) – The number of images to be included in the dataset. (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> cifar100_dataset_dir = "/path/to/cifar100_dataset_directory" >>> >>> # 1) Get all samples from CIFAR100 dataset in sequence >>> dataset = ds.Cifar100Dataset(dataset_dir=cifar100_dataset_dir, shuffle=False) >>> >>> # 2) Randomly select 350 samples from CIFAR100 dataset >>> dataset = ds.Cifar100Dataset(dataset_dir=cifar100_dataset_dir, num_samples=350, shuffle=True) >>> >>> # In CIFAR100 dataset, each dictionary has 3 keys: "image", "fine_label" and "coarse_label"
-
class
tinyms.data.Cifar10Dataset(dataset_dir, usage=None, num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing Cifar10 dataset.
The generated dataset has two columns [‘image’, ‘label’]. The type of the image tensor is uint8. The label is a scalar uint32 tensor. This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of Cifar10 dataset.
@techreport{Krizhevsky09, author = {Alex Krizhevsky}, title = {Learning multiple layers of features from tiny images}, institution = {}, year = {2009}, howpublished = {http://www.cs.toronto.edu/~kriz/cifar.html}, description = {The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
usage (str, optional) – Usage of this dataset, can be “train”, “test” or “all” . “train” will read from 50,000 train samples, “test” will read from 10,000 test samples, “all” will read from all 60,000 samples. (default=None, all samples)
num_samples (int, optional) – The number of images to be included in the dataset. (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> cifar10_dataset_dir = "/path/to/cifar10_dataset_directory" >>> >>> # 1) Get all samples from CIFAR10 dataset in sequence >>> dataset = ds.Cifar10Dataset(dataset_dir=cifar10_dataset_dir, shuffle=False) >>> >>> # 2) Randomly select 350 samples from CIFAR10 dataset >>> dataset = ds.Cifar10Dataset(dataset_dir=cifar10_dataset_dir, num_samples=350, shuffle=True) >>> >>> # 3) Get samples from CIFAR10 dataset for shard 0 in a 2-way distributed training >>> dataset = ds.Cifar10Dataset(dataset_dir=cifar10_dataset_dir, num_shards=2, shard_id=0) >>> >>> # In CIFAR10 dataset, each dictionary has keys "image" and "label"
-
class
tinyms.data.CLUEDataset(dataset_files, task='AFQMC', usage='train', num_samples=None, num_parallel_workers=None, shuffle=<Shuffle.GLOBAL: 'global'>, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads and parses CLUE datasets. CLUE, the Chinese Language Understanding Evaluation Benchmark, is a collection of datasets, baselines, pre-trained models, corpus and leaderboard. Supported CLUE classification tasks: ‘AFQMC’, ‘TNEWS’, ‘IFLYTEK’, ‘CMNLI’, ‘WSC’ and ‘CSL’.
Citation of CLUE dataset.
@article{CLUEbenchmark, title = {CLUE: A Chinese Language Understanding Evaluation Benchmark}, author = {Liang Xu, Xuanwei Zhang, Lu Li, Hai Hu, Chenjie Cao, Weitang Liu, Junyi Li, Yudong Li, Kai Sun, Yechen Xu, Yiming Cui, Cong Yu, Qianqian Dong, Yin Tian, Dian Yu, Bo Shi, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Zhenzhong Lan}, journal = {arXiv preprint arXiv:2004.05986}, year = {2020}, howpublished = {https://github.com/CLUEbenchmark/CLUE}, description = {CLUE, a Chinese Language Understanding Evaluation benchmark. It contains eight different tasks, including single-sentence classification, sentence pair classification, and machine reading comprehension.} }
- Parameters
dataset_files (Union[str, list[str]]) – String or list of files to be read or glob strings to search for a pattern of files. The list will be sorted in a lexicographical order.
task (str, optional) – The kind of task, one of ‘AFQMC’, ‘TNEWS’, ‘IFLYTEK’, ‘CMNLI’, ‘WSC’ and ‘CSL’. (default=AFQMC).
usage (str, optional) – Need train, test or eval data (default=”train”).
num_samples (int, optional) – Number of samples (rows) to read (default=None, reads the full dataset).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (Union[bool, Shuffle level], optional) –
Perform reshuffling of the data every epoch (default=Shuffle.GLOBAL). If shuffle is False, no shuffling will be performed; If shuffle is True, the behavior is the same as setting shuffle to be Shuffle.GLOBAL Otherwise, there are two levels of shuffling:
Shuffle.GLOBAL: Shuffle both the files and samples.
Shuffle.FILES: Shuffle files only.
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
Examples
>>> clue_dataset_dir = ["/path/to/clue_dataset_file"] # contains 1 or multiple clue files >>> dataset = ds.CLUEDataset(dataset_files=clue_dataset_dir, task='AFQMC', usage='train')
-
class
tinyms.data.CocoDataset(dataset_dir, annotation_file, task='Detection', num_samples=None, num_parallel_workers=None, shuffle=None, decode=False, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing COCO dataset.
CocoDataset supports four kinds of tasks, which are Object Detection, Keypoint Detection, Stuff Segmentation and Panoptic Segmentation of 2017 Train/Val/Test dataset.
The generated dataset has multi-columns :
task=’Detection’, column: [[‘image’, dtype=uint8], [‘bbox’, dtype=float32], [‘category_id’, dtype=uint32], [‘iscrowd’, dtype=uint32]].
task=’Stuff’, column: [[‘image’, dtype=uint8], [‘segmentation’,dtype=float32], [‘iscrowd’,dtype=uint32]].
task=’Keypoint’, column: [[‘image’, dtype=uint8], [‘keypoints’, dtype=float32], [‘num_keypoints’, dtype=uint32]].
task=’Panoptic’, column: [[‘image’, dtype=uint8], [‘bbox’, dtype=float32], [‘category_id’, dtype=uint32], [‘iscrowd’, dtype=uint32], [‘area’, dtype=uint32]].
This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. CocoDataset doesn’t support PKSampler. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of Coco dataset.
@article{DBLP:journals/corr/LinMBHPRDZ14, author = {Tsung{-}Yi Lin and Michael Maire and Serge J. Belongie and Lubomir D. Bourdev and Ross B. Girshick and James Hays and Pietro Perona and Deva Ramanan and Piotr Doll{'{a}}r and C. Lawrence Zitnick}, title = {Microsoft {COCO:} Common Objects in Context}, journal = {CoRR}, volume = {abs/1405.0312}, year = {2014}, url = {http://arxiv.org/abs/1405.0312}, archivePrefix = {arXiv}, eprint = {1405.0312}, timestamp = {Mon, 13 Aug 2018 16:48:13 +0200}, biburl = {https://dblp.org/rec/journals/corr/LinMBHPRDZ14.bib}, bibsource = {dblp computer science bibliography, https://dblp.org}, description = {COCO is a large-scale object detection, segmentation, and captioning dataset. It contains 91 common object categories with 82 of them having more than 5,000 labeled instances. In contrast to the popular ImageNet dataset, COCO has fewer categories but more instances per category.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
annotation_file (str) – Path to the annotation JSON.
task (str) – Set the task type for reading COCO data. Supported task types: ‘Detection’, ‘Stuff’, ‘Panoptic’ and ‘Keypoint’ (default=’Detection’).
num_samples (int, optional) – The number of images to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the configuration file).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
decode (bool, optional) – Decode the images after reading (default=False).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
RuntimeError – If parse JSON file failed.
ValueError – If task is not in [‘Detection’, ‘Stuff’, ‘Panoptic’, ‘Keypoint’].
ValueError – If annotation_file is not exist.
ValueError – If dataset_dir is not exist.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> coco_dataset_dir = "/path/to/coco_dataset_directory/images" >>> coco_annotation_file = "/path/to/coco_dataset_directory/annotation_file" >>> >>> # 1) Read COCO data for Detection task >>> dataset = ds.CocoDataset(dataset_dir=coco_dataset_dir, ... annotation_file=coco_annotation_file, ... task='Detection') >>> >>> # 2) Read COCO data for Stuff task >>> dataset = ds.CocoDataset(dataset_dir=coco_dataset_dir, ... annotation_file=coco_annotation_file, ... task='Stuff') >>> >>> # 3) Read COCO data for Panoptic task >>> dataset = ds.CocoDataset(dataset_dir=coco_dataset_dir, ... annotation_file=coco_annotation_file, ... task='Panoptic') >>> >>> # 4) Read COCO data for Keypoint task >>> dataset = ds.CocoDataset(dataset_dir=coco_dataset_dir, ... annotation_file=coco_annotation_file, ... task='Keypoint') >>> >>> # In COCO dataset, each dictionary has keys "image" and "annotation"
-
class
tinyms.data.CSVDataset(dataset_files, field_delim=', ', column_defaults=None, column_names=None, num_samples=None, num_parallel_workers=None, shuffle=<Shuffle.GLOBAL: 'global'>, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads and parses comma-separated values (CSV) datasets.
- Parameters
dataset_files (Union[str, list[str]]) – String or list of files to be read or glob strings to search for a pattern of files. The list will be sorted in a lexicographical order.
field_delim (str, optional) – A string that indicates the char delimiter to separate fields (default=’,’).
column_defaults (list, optional) – List of default values for the CSV field (default=None). Each item in the list is either a valid type (float, int, or string). If this is not provided, treats all columns as string type.
column_names (list[str], optional) – List of column names of the dataset (default=None). If this is not provided, infers the column_names from the first row of CSV file.
num_samples (int, optional) – Number of samples (rows) to read (default=None, reads the full dataset).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (Union[bool, Shuffle level], optional) –
Perform reshuffling of the data every epoch (default=Shuffle.GLOBAL). If shuffle is False, no shuffling will be performed; If shuffle is True, the behavior is the same as setting shuffle to be Shuffle.GLOBAL Otherwise, there are two levels of shuffling:
Shuffle.GLOBAL: Shuffle both the files and samples.
Shuffle.FILES: Shuffle files only.
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
Examples
>>> csv_dataset_dir = ["/path/to/csv_dataset_file"] # contains 1 or multiple csv files >>> dataset = ds.CSVDataset(dataset_files=csv_dataset_dir, column_names=['col1', 'col2', 'col3', 'col4'])
-
class
tinyms.data.GeneratorDataset(source, column_names=None, column_types=None, schema=None, num_samples=None, num_parallel_workers=1, shuffle=None, sampler=None, num_shards=None, shard_id=None, python_multiprocessing=True)[source]¶ A source dataset that generates data from Python by invoking Python data source each epoch.
This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
- Parameters
source (Union[Callable, Iterable, Random Accessible]) – A generator callable object, an iterable Python object or a random accessible Python object. Callable source is required to return a tuple of NumPy arrays as a row of the dataset on source().next(). Iterable source is required to return a tuple of NumPy arrays as a row of the dataset on iter(source).next(). Random accessible source is required to return a tuple of NumPy arrays as a row of the dataset on source[idx].
column_names (Union[str, list[str]], optional) – List of column names of the dataset (default=None). Users are required to provide either column_names or schema.
column_types (list[mindspore.dtype], optional) – List of column data types of the dataset (default=None). If provided, sanity check will be performed on generator output.
schema (Union[Schema, str], optional) – Path to the JSON schema file or schema object (default=None). Users are required to provide either column_names or schema. If both are provided, schema will be used.
num_samples (int, optional) – The number of samples to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of subprocesses used to fetch the dataset in parallel (default=1).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset. Random accessible input is required. (default=None, expected order behavior shown in the table).
sampler (Union[Sampler, Iterable], optional) – Object used to choose samples from the dataset. Random accessible input is required (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). Random accessible input is required. When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument must be specified only when num_shards is also specified. Random accessible input is required.
python_multiprocessing (bool, optional) – Parallelize Python operations with multiple worker process. This option could be beneficial if the Python operation is computational heavy (default=True).
Examples
>>> import numpy as np >>> >>> # 1) Multidimensional generator function as callable input. >>> def generator_multidimensional(): ... for i in range(64): ... yield (np.array([[i, i + 1], [i + 2, i + 3]]),) >>> >>> dataset = ds.GeneratorDataset(source=generator_multidimensional, column_names=["multi_dimensional_data"]) >>> >>> # 2) Multi-column generator function as callable input. >>> def generator_multi_column(): ... for i in range(64): ... yield np.array([i]), np.array([[i, i + 1], [i + 2, i + 3]]) >>> >>> dataset = ds.GeneratorDataset(source=generator_multi_column, column_names=["col1", "col2"]) >>> >>> # 3) Iterable dataset as iterable input. >>> class MyIterable: ... def __init__(self): ... self._index = 0 ... self._data = np.random.sample((5, 2)) ... self._label = np.random.sample((5, 1)) ... ... def __next__(self): ... if self._index >= len(self._data): ... raise StopIteration ... else: ... item = (self._data[self._index], self._label[self._index]) ... self._index += 1 ... return item ... ... def __iter__(self): ... self._index = 0 ... return self ... ... def __len__(self): ... return len(self._data) >>> >>> dataset = ds.GeneratorDataset(source=MyIterable(), column_names=["data", "label"]) >>> >>> # 4) Random accessible dataset as random accessible input. >>> class MyAccessible: ... def __init__(self): ... self._data = np.random.sample((5, 2)) ... self._label = np.random.sample((5, 1)) ... ... def __getitem__(self, index): ... return self._data[index], self._label[index] ... ... def __len__(self): ... return len(self._data) >>> >>> dataset = ds.GeneratorDataset(source=MyAccessible(), column_names=["data", "label"]) >>> >>> # list, dict, tuple of Python is also random accessible >>> dataset = ds.GeneratorDataset(source=[(np.array(0),), (np.array(1),), (np.array(2),)], column_names=["col"])
-
class
tinyms.data.GraphData(dataset_file, num_parallel_workers=None, working_mode='local', hostname='127.0.0.1', port=50051, num_client=1, auto_shutdown=True)[source]¶ Reads the graph dataset used for GNN training from the shared file and database.
- Parameters
dataset_file (str) – One of file names in the dataset.
num_parallel_workers (int, optional) – Number of workers to process the dataset in parallel (default=None).
working_mode (str, optional) –
Set working mode, now supports ‘local’/’client’/’server’ (default=’local’).
’local’, used in non-distributed training scenarios.
’client’, used in distributed training scenarios. The client does not load data, but obtains data from the server.
’server’, used in distributed training scenarios. The server loads the data and is available to the client.
hostname (str, optional) – Hostname of the graph data server. This parameter is only valid when working_mode is set to ‘client’ or ‘server’ (default=’127.0.0.1’).
port (int, optional) – Port of the graph data server. The range is 1024-65535. This parameter is only valid when working_mode is set to ‘client’ or ‘server’ (default=50051).
num_client (int, optional) – Maximum number of clients expected to connect to the server. The server will allocate resources according to this parameter. This parameter is only valid when working_mode is set to ‘server’ (default=1).
auto_shutdown (bool, optional) – Valid when working_mode is set to ‘server’, when the number of connected clients reaches num_client and no client is being connected, the server automatically exits (default=True).
Examples
>>> graph_dataset_dir = "/path/to/graph_dataset_file" >>> graph_dataset = ds.GraphData(dataset_file=graph_dataset_dir, num_parallel_workers=2) >>> nodes = graph_dataset.get_all_nodes(node_type=1) >>> features = graph_dataset.get_node_feature(node_list=nodes, feature_types=[1])
-
get_all_edges(edge_type)[source]¶ Get all edges in the graph.
- Parameters
edge_type (int) – Specify the type of edge.
- Returns
numpy.ndarray, array of edges.
Examples
>>> edges = graph_dataset.get_all_edges(edge_type=0)
- Raises
TypeError – If edge_type is not integer.
-
get_all_neighbors(node_list, neighbor_type)[source]¶ Get neighbor_type neighbors of the nodes in node_list.
- Parameters
node_list (Union[list, numpy.ndarray]) – The given list of nodes.
neighbor_type (int) – Specify the type of neighbor.
- Returns
numpy.ndarray, array of neighbors.
Examples
>>> nodes = graph_dataset.get_all_nodes(node_type=1) >>> neighbors = graph_dataset.get_all_neighbors(node_list=nodes, neighbor_type=2)
-
get_all_nodes(node_type)[source]¶ Get all nodes in the graph.
- Parameters
node_type (int) – Specify the type of node.
- Returns
numpy.ndarray, array of nodes.
Examples
>>> nodes = graph_dataset.get_all_nodes(node_type=1)
- Raises
TypeError – If node_type is not integer.
-
get_edge_feature(edge_list, feature_types)[source]¶ Get feature_types feature of the edges in edge_list.
- Parameters
edge_list (Union[list, numpy.ndarray]) – The given list of edges.
feature_types (Union[list, numpy.ndarray]) – The given list of feature types.
- Returns
numpy.ndarray, array of features.
Examples
>>> edges = graph_dataset.get_all_edges(edge_type=0) >>> features = graph_dataset.get_edge_feature(edge_list=edges, feature_types=[1])
-
get_neg_sampled_neighbors(node_list, neg_neighbor_num, neg_neighbor_type)[source]¶ Get neg_neighbor_type negative sampled neighbors of the nodes in node_list.
- Parameters
node_list (Union[list, numpy.ndarray]) – The given list of nodes.
neg_neighbor_num (int) – Number of neighbors sampled.
neg_neighbor_type (int) – Specify the type of negative neighbor.
- Returns
numpy.ndarray, array of neighbors.
Examples
>>> nodes = graph_dataset.get_all_nodes(node_type=1) >>> neg_neighbors = graph_dataset.get_neg_sampled_neighbors(node_list=nodes, neg_neighbor_num=5, ... neg_neighbor_type=2)
-
get_node_feature(node_list, feature_types)[source]¶ Get feature_types feature of the nodes in node_list.
- Parameters
node_list (Union[list, numpy.ndarray]) – The given list of nodes.
feature_types (Union[list, numpy.ndarray]) – The given list of feature types.
- Returns
numpy.ndarray, array of features.
Examples
>>> nodes = graph_dataset.get_all_nodes(node_type=1) >>> features = graph_dataset.get_node_feature(node_list=nodes, feature_types=[2, 3])
-
get_nodes_from_edges(edge_list)[source]¶ Get nodes from the edges.
- Parameters
edge_list (Union[list, numpy.ndarray]) – The given list of edges.
- Returns
numpy.ndarray, array of nodes.
- Raises
TypeError – If edge_list is not list or ndarray.
-
get_sampled_neighbors(node_list, neighbor_nums, neighbor_types, strategy=<SamplingStrategy.RANDOM: 0>)[source]¶ Get sampled neighbor information.
The api supports multi-hop neighbor sampling. That is, the previous sampling result is used as the input of next-hop sampling. A maximum of 6-hop are allowed.
The sampling result is tiled into a list in the format of [input node, 1-hop sampling result, 2-hop samling result …]
- Parameters
node_list (Union[list, numpy.ndarray]) – The given list of nodes.
neighbor_nums (Union[list, numpy.ndarray]) – Number of neighbors sampled per hop.
neighbor_types (Union[list, numpy.ndarray]) – Neighbor type sampled per hop.
strategy (SamplingStrategy, optional) –
Sampling strategy (default=SamplingStrategy.RANDOM). It can be any of [SamplingStrategy.RANDOM, SamplingStrategy.EDGE_WEIGHT].
SamplingStrategy.RANDOM, random sampling with replacement.
SamplingStrategy.EDGE_WEIGHT, sampling with edge weight as probability.
- Returns
numpy.ndarray, array of neighbors.
Examples
>>> nodes = graph_dataset.get_all_nodes(node_type=1) >>> neighbors = graph_dataset.get_sampled_neighbors(node_list=nodes, neighbor_nums=[2, 2], ... neighbor_types=[2, 1])
-
graph_info()[source]¶ Get the meta information of the graph, including the number of nodes, the type of nodes, the feature information of nodes, the number of edges, the type of edges, and the feature information of edges.
- Returns
dict, meta information of the graph. The key is node_type, edge_type, node_num, edge_num, node_feature_type and edge_feature_type.
-
random_walk(target_nodes, meta_path, step_home_param=1.0, step_away_param=1.0, default_node=-1)[source]¶ Random walk in nodes.
- Parameters
step_home_param (float, optional) – return hyper parameter in node2vec algorithm (Default = 1.0).
step_away_param (float, optional) – in out hyper parameter in node2vec algorithm (Default = 1.0).
default_node (int, optional) – default node if no more neighbors found (Default = -1). A default value of -1 indicates that no node is given.
- Returns
numpy.ndarray, array of nodes.
Examples
>>> nodes = graph_dataset.get_all_nodes(node_type=1) >>> walks = graph_dataset.random_walk(target_nodes=nodes, meta_path=[2, 1, 2])
-
class
tinyms.data.ImageFolderDataset(dataset_dir, num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, extensions=None, class_indexing=None, decode=False, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads images from a tree of directories.
All images within one folder have the same label. The generated dataset has two columns [‘image’, ‘label’]. The shape of the image column is [image_size] if decode flag is False, or [H,W,C] otherwise. The type of the image tensor is uint8. The label is a scalar int32 tensor. This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
num_samples (int, optional) – The number of images to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, set in the config).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
extensions (list[str], optional) – List of file extensions to be included in the dataset (default=None).
class_indexing (dict, optional) – A str-to-int mapping from folder name to index (default=None, the folder names will be sorted alphabetically and each class will be given a unique index starting from 0).
decode (bool, optional) – Decode the images after reading (default=False).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
RuntimeError – If class_indexing is not a dictionary.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> image_folder_dataset_dir = "/path/to/image_folder_dataset_directory" >>> >>> # 1) Read all samples (image files) in image_folder_dataset_dir with 8 threads >>> dataset = ds.ImageFolderDataset(dataset_dir=image_folder_dataset_dir, ... num_parallel_workers=8) >>> >>> # 2) Read all samples (image files) from folder cat and folder dog with label 0 and 1 >>> dataset = ds.ImageFolderDataset(dataset_dir=image_folder_dataset_dir, ... class_indexing={"cat":0, "dog":1}) >>> >>> # 3) Read all samples (image files) in image_folder_dataset_dir with extensions .JPEG and .png (case sensitive) >>> dataset = ds.ImageFolderDataset(dataset_dir=image_folder_dataset_dir, ... extensions=[".JPEG", ".png"])
-
class
tinyms.data.ManifestDataset(dataset_file, usage='train', num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, class_indexing=None, decode=False, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading images from a Manifest file.
The generated dataset has two columns [‘image’, ‘label’]. The shape of the image column is [image_size] if decode flag is False, or [H,W,C] otherwise. The type of the image tensor is uint8. The label is a scalar uint64 tensor. This dataset can take in a sampler. sampler and shuffle are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using sampler and shuffle¶ Parameter sampler
Parameter shuffle
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
- Parameters
dataset_file (str) – File to be read.
usage (str, optional) – Acceptable usages include “train”, “eval” and “inference” (default=”train”).
num_samples (int, optional) – The number of images to be included in the dataset. (default=None, will include all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, will use value set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
class_indexing (dict, optional) – A str-to-int mapping from label name to index (default=None, the folder names will be sorted alphabetically and each class will be given a unique index starting from 0).
decode (bool, optional) – decode the images after reading (default=False).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, num_samples reflects the max number of samples per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
RuntimeError – If class_indexing is not a dictionary.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> manifest_dataset_dir = "/path/to/manifest_dataset_file" >>> >>> # 1) Read all samples specified in manifest_dataset_dir dataset with 8 threads for training >>> dataset = ds.ManifestDataset(dataset_file=manifest_dataset_dir, usage="train", num_parallel_workers=8) >>> >>> # 2) Read samples (specified in manifest_file.manifest) for shard 0 in a 2-way distributed training setup >>> dataset = ds.ManifestDataset(dataset_file=manifest_dataset_dir, num_shards=2, shard_id=0)
-
class
tinyms.data.MindDataset(dataset_file, columns_list=None, num_parallel_workers=None, shuffle=None, num_shards=None, shard_id=None, sampler=None, padded_sample=None, num_padded=None, num_samples=None)[source]¶ A source dataset for reading and parsing MindRecord dataset.
- Parameters
dataset_file (Union[str, list[str]]) – If dataset_file is a str, it represents for a file name of one component of a mindrecord source, other files with identical source in the same path will be found and loaded automatically. If dataset_file is a list, it represents for a list of dataset files to be read directly.
columns_list (list[str], optional) – List of columns to be read (default=None).
num_parallel_workers (int, optional) – The number of readers (default=None).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset (default=None, performs shuffle).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, sampler is exclusive with shuffle and block_reader). Support list: SubsetRandomSampler, PkSampler, RandomSampler, SequentialSampler, DistributedSampler.
padded_sample (dict, optional) – Samples will be appended to dataset, where keys are the same as column_list.
num_padded (int, optional) – Number of padding samples. Dataset size plus num_padded should be divisible by num_shards.
num_samples (int, optional) – The number of samples to be included in the dataset (default=None, all samples).
- Raises
ValueError – If num_shards is specified but shard_id is None.
ValueError – If shard_id is specified but num_shards is None.
Examples
>>> mind_dataset_dir = ["/path/to/mind_dataset_file"] # contains 1 or multiple MindRecord files >>> dataset = ds.MindDataset(dataset_file=mind_dataset_dir)
-
class
tinyms.data.MnistDataset(dataset_dir, usage=None, num_samples=None, num_parallel_workers=None, shuffle=None, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing the MNIST dataset.
The generated dataset has two columns [‘image’, ‘label’]. The type of the image tensor is uint8. The label is a scalar uint32 tensor. This dataset can take in a sampler. sampler and shuffle are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter sampler
Parameter shuffle
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of Mnist dataset.
@article{lecun2010mnist, title = {MNIST handwritten digit database}, author = {LeCun, Yann and Cortes, Corinna and Burges, CJ}, journal = {ATT Labs [Online]}, volume = {2}, year = {2010}, howpublished = {http://yann.lecun.com/exdb/mnist}, description = {The MNIST database of handwritten digits has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
usage (str, optional) – Usage of this dataset, can be “train”, “test” or “all” . “train” will read from 60,000 train samples, “test” will read from 10,000 test samples, “all” will read from all 70,000 samples. (default=None, will read all samples)
num_samples (int, optional) – The number of images to be included in the dataset (default=None, will read all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, will use value set in the config).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, num_samples reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
- Raises
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> mnist_dataset_dir = "/path/to/mnist_dataset_directory" >>> >>> # Read 3 samples from MNIST dataset >>> dataset = ds.MnistDataset(dataset_dir=mnist_dataset_dir, num_samples=3) >>> >>> # Note: In mnist_dataset dataset, each dictionary has keys "image" and "label"
-
class
tinyms.data.NumpySlicesDataset(data, column_names=None, num_samples=None, num_parallel_workers=1, shuffle=None, sampler=None, num_shards=None, shard_id=None)[source]¶ Creates a dataset with given data slices, mainly for loading Python data into dataset.
This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
- Parameters
data (Union[list, tuple, dict]) – list, tuple, dict and other NumPy formats. Input data will be sliced along the first dimension and generate additional rows, if input is list, there will be one column in each row, otherwise there tends to be multi columns. Large data is not recommended to be loaded in this way as data is loading into memory.
column_names (list[str], optional) – List of column names of the dataset (default=None). If column_names is not provided, when data is dict, column_names will be its keys, otherwise it will be like column_0, column_1 …
num_samples (int, optional) – The number of samples to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of subprocesses used to fetch the dataset in parallel (default=1).
shuffle (bool, optional) – Whether or not to perform shuffle on the dataset. Random accessible input is required. (default=None, expected order behavior shown in the table).
sampler (Union[Sampler, Iterable], optional) – Object used to choose samples from the dataset. Random accessible input is required (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). Random accessible input is required. When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument must be specified only when num_shards is also specified. Random accessible input is required.
Examples
>>> # 1) Input data can be a list >>> data = [1, 2, 3] >>> dataset = ds.NumpySlicesDataset(data=data, column_names=["column_1"]) >>> >>> # 2) Input data can be a dictionary, and column_names will be its keys >>> data = {"a": [1, 2], "b": [3, 4]} >>> dataset = ds.NumpySlicesDataset(data=data) >>> >>> # 3) Input data can be a tuple of lists (or NumPy arrays), each tuple element refers to data in each column >>> data = ([1, 2], [3, 4], [5, 6]) >>> dataset = ds.NumpySlicesDataset(data=data, column_names=["column_1", "column_2", "column_3"]) >>> >>> # 4) Load data from CSV file >>> import pandas as pd >>> df = pd.read_csv(filepath_or_buffer=csv_dataset_dir[0]) >>> dataset = ds.NumpySlicesDataset(data=dict(df), shuffle=False)
-
class
tinyms.data.PaddedDataset(padded_samples)[source]¶ Creates a dataset with filler data provided by user. Mainly used to add to the original data set and assign it to the corresponding shard.
- Parameters
- Raises
TypeError – If padded_samples is not an instance of list.
TypeError – If the element of padded_samples is not an instance of dict.
ValueError – If the padded_samples is empty.
Examples
>>> import numpy as np >>> data = [{'image': np.zeros(1, np.uint8)}, {'image': np.zeros(2, np.uint8)}] >>> dataset = ds.PaddedDataset(padded_samples=data)
-
class
tinyms.data.TextFileDataset(dataset_files, num_samples=None, num_parallel_workers=None, shuffle=<Shuffle.GLOBAL: 'global'>, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset that reads and parses datasets stored on disk in text format. The generated dataset has one column [‘text’].
- Parameters
dataset_files (Union[str, list[str]]) – String or list of files to be read or glob strings to search for a pattern of files. The list will be sorted in a lexicographical order.
num_samples (int, optional) – Number of samples (rows) to read (default=None, reads the full dataset).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (Union[bool, Shuffle level], optional) –
Perform reshuffling of the data every epoch (default=Shuffle.GLOBAL). If shuffle is False, no shuffling will be performed; If shuffle is True, the behavior is the same as setting shuffle to be Shuffle.GLOBAL Otherwise, there are two levels of shuffling:
Shuffle.GLOBAL: Shuffle both the files and samples.
Shuffle.FILES: Shuffle files only.
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
Examples
>>> text_file_dataset_dir = ["/path/to/text_file_dataset_file"] # contains 1 or multiple text files >>> dataset = ds.TextFileDataset(dataset_files=text_file_dataset_dir)
-
class
tinyms.data.TFRecordDataset(dataset_files, schema=None, columns_list=None, num_samples=None, num_parallel_workers=None, shuffle=<Shuffle.GLOBAL: 'global'>, num_shards=None, shard_id=None, shard_equal_rows=False, cache=None)[source]¶ A source dataset for reading and parsing datasets stored on disk in TFData format.
- Parameters
dataset_files (Union[str, list[str]]) – String or list of files to be read or glob strings to search for a pattern of files. The list will be sorted in a lexicographical order.
schema (Union[str, Schema], optional) – Path to the JSON schema file or schema object (default=None). If the schema is not provided, the meta data from the TFData file is considered the schema.
columns_list (list[str], optional) – List of columns to be read (default=None, read all columns)
num_samples (int, optional) – Number of samples (rows) to read (default=None). If num_samples is None and numRows(parsed from schema) does not exist, read the full dataset; If num_samples is None and numRows(parsed from schema) is greater than 0, read numRows rows; If both num_samples and numRows(parsed from schema) are greater than 0, read num_samples rows.
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (Union[bool, Shuffle level], optional) –
Perform reshuffling of the data every epoch (default=Shuffle.GLOBAL). If shuffle is False, no shuffling will be performed; If shuffle is True, the behavior is the same as setting shuffle to be Shuffle.GLOBAL Otherwise, there are two levels of shuffling:
Shuffle.GLOBAL: Shuffle both the files and samples.
Shuffle.FILES: Shuffle files only.
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
shard_equal_rows (bool, optional) – Get equal rows for all shards(default=False). If shard_equal_rows is false, number of rows of each shard may be not equal. This argument should only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
Examples
>>> import mindspore.common.dtype as mstype >>> >>> tfrecord_dataset_dir = ["/path/to/tfrecord_dataset_file"] # contains 1 or multiple TFRecord files >>> tfrecord_schema_file = "/path/to/tfrecord_schema_file" >>> >>> # 1) Get all rows from tfrecord_dataset_dir with no explicit schema. >>> # The meta-data in the first row will be used as a schema. >>> dataset = ds.TFRecordDataset(dataset_files=tfrecord_dataset_dir) >>> >>> # 2) Get all rows from tfrecord_dataset_dir with user-defined schema. >>> schema = ds.Schema() >>> schema.add_column(name='col_1d', de_type=mstype.int64, shape=[2]) >>> dataset = ds.TFRecordDataset(dataset_files=tfrecord_dataset_dir, schema=schema) >>> >>> # 3) Get all rows from tfrecord_dataset_dir with schema file. >>> dataset = ds.TFRecordDataset(dataset_files=tfrecord_dataset_dir, schema=tfrecord_schema_file)
-
class
tinyms.data.VOCDataset(dataset_dir, task='Segmentation', usage='train', class_indexing=None, num_samples=None, num_parallel_workers=None, shuffle=None, decode=False, sampler=None, num_shards=None, shard_id=None, cache=None)[source]¶ A source dataset for reading and parsing VOC dataset.
The generated dataset has multiple columns :
task=’Detection’, column: [[‘image’, dtype=uint8], [‘bbox’, dtype=float32], [‘label’, dtype=uint32], [‘difficult’, dtype=uint32], [‘truncate’, dtype=uint32]].
task=’Segmentation’, column: [[‘image’, dtype=uint8], [‘target’,dtype=uint8]].
This dataset can take in a sampler. ‘sampler’ and ‘shuffle’ are mutually exclusive. The table below shows what input arguments are allowed and their expected behavior.
Expected Order Behavior of Using ‘sampler’ and ‘shuffle’¶ Parameter ‘sampler’
Parameter ‘shuffle’
Expected Order Behavior
None
None
random order
None
True
random order
None
False
sequential order
Sampler object
None
order defined by sampler
Sampler object
True
not allowed
Sampler object
False
not allowed
Citation of VOC dataset.
@article{Everingham10, author = {Everingham, M. and Van~Gool, L. and Williams, C. K. I. and Winn, J. and Zisserman, A.}, title = {The Pascal Visual Object Classes (VOC) Challenge}, journal = {International Journal of Computer Vision}, volume = {88}, year = {2010}, number = {2}, month = {jun}, pages = {303--338}, biburl = {http://host.robots.ox.ac.uk/pascal/VOC/pubs/everingham10.html#bibtex}, howpublished = {http://host.robots.ox.ac.uk/pascal/VOC/voc{year}/index.html}, description = {The PASCAL Visual Object Classes (VOC) challenge is a benchmark in visual object category recognition and detection, providing the vision and machine learning communities with a standard dataset of images and annotation, and standard evaluation procedures.} }
- Parameters
dataset_dir (str) – Path to the root directory that contains the dataset.
task (str) – Set the task type of reading voc data, now only support “Segmentation” or “Detection” (default=”Segmentation”).
usage (str) – The type of data list text file to be read (default=”train”).
class_indexing (dict, optional) – A str-to-int mapping from label name to index, only valid in “Detection” task (default=None, the folder names will be sorted alphabetically and each class will be given a unique index starting from 0).
num_samples (int, optional) – The number of images to be included in the dataset (default=None, all images).
num_parallel_workers (int, optional) – Number of workers to read the data (default=None, number set in the config).
shuffle (bool, optional) – Whether to perform shuffle on the dataset (default=None, expected order behavior shown in the table).
decode (bool, optional) – Decode the images after reading (default=False).
sampler (Sampler, optional) – Object used to choose samples from the dataset (default=None, expected order behavior shown in the table).
num_shards (int, optional) – Number of shards that the dataset will be divided into (default=None). When this argument is specified, ‘num_samples’ reflects the max sample number of per shard.
shard_id (int, optional) – The shard ID within num_shards (default=None). This argument can only be specified when num_shards is also specified.
cache (DatasetCache, optional) – Use tensor caching service to speed up dataset processing. (default=None, which means no cache is used).
- Raises
RuntimeError – If xml of Annotations is an invalid format.
RuntimeError – If xml of Annotations loss attribution of “object”.
RuntimeError – If xml of Annotations loss attribution of “bndbox”.
RuntimeError – If sampler and shuffle are specified at the same time.
RuntimeError – If sampler and sharding are specified at the same time.
RuntimeError – If num_shards is specified but shard_id is None.
RuntimeError – If shard_id is specified but num_shards is None.
ValueError – If task is not equal ‘Segmentation’ or ‘Detection’.
ValueError – If task equal ‘Segmentation’ but class_indexing is not None.
ValueError – If txt related to mode is not exist.
ValueError – If shard_id is invalid (< 0 or >= num_shards).
Examples
>>> voc_dataset_dir = "/path/to/voc_dataset_directory" >>> >>> # 1) Read VOC data for segmentatation training >>> dataset = ds.VOCDataset(dataset_dir=voc_dataset_dir, task="Segmentation", usage="train") >>> >>> # 2) Read VOC data for detection training >>> dataset = ds.VOCDataset(dataset_dir=voc_dataset_dir, task="Detection", usage="train") >>> >>> # 3) Read all VOC dataset samples in voc_dataset_dir with 8 threads in random order >>> dataset = ds.VOCDataset(dataset_dir=voc_dataset_dir, task="Detection", usage="train", ... num_parallel_workers=8) >>> >>> # 4) Read then decode all VOC dataset samples in voc_dataset_dir in sequence >>> dataset = ds.VOCDataset(dataset_dir=voc_dataset_dir, task="Detection", usage="train", ... decode=True, shuffle=False) >>> >>> # In VOC dataset, if task='Segmentation', each dictionary has keys "image" and "target" >>> # In VOC dataset, if task='Detection', each dictionary has keys "image" and "annotation"
-
class
tinyms.data.DistributedSampler(dataset_size, num_replicas=None, rank=None, shuffle=True)[source] Distributed sampler.
-
class
tinyms.data.PKSampler(num_val, num_class=None, shuffle=False, class_column='label', num_samples=None)[source]¶ Samples K elements for each P class in the dataset.
- Parameters
num_val (int) – Number of elements to sample for each class.
num_class (int, optional) – Number of classes to sample (default=None, all classes). The parameter does not supported to specify currently.
shuffle (bool, optional) – If True, the class IDs are shuffled (default=False).
class_column (str, optional) – Name of column with class labels for MindDataset (default=’label’).
num_samples (int, optional) – The number of samples to draw (default=None, all elements).
Examples
>>> # creates a PKSampler that will get 3 samples from every class. >>> sampler = ds.PKSampler(3) >>> dataset = ds.ImageFolderDataset(image_folder_dataset_dir, ... num_parallel_workers=8, ... sampler=sampler)
- Raises
TypeError – If num_val is not a positive value.
TypeError – If shuffle is not a boolean value.
TypeError – If class_column is not a str value.
TypeError – If num_samples is not an integer value.
NotImplementedError – If num_class is not None.
RuntimeError – If num_val is not a positive value.
RuntimeError – If num_samples is a negative value.
-
class
tinyms.data.RandomSampler(replacement=False, num_samples=None)[source]¶ Samples the elements randomly.
- Parameters
Examples
>>> # creates a RandomSampler >>> sampler = ds.RandomSampler() >>> dataset = ds.ImageFolderDataset(image_folder_dataset_dir, ... num_parallel_workers=8, ... sampler=sampler)
- Raises
TypeError – If replacement is not a boolean value.
TypeError – If num_samples is not an integer value.
RuntimeError – If num_samples is a negative value.
-
class
tinyms.data.SequentialSampler(start_index=None, num_samples=None)[source]¶ Samples the dataset elements sequentially, same as not having a sampler.
- Parameters
Examples
>>> # creates a SequentialSampler >>> sampler = ds.SequentialSampler() >>> dataset = ds.ImageFolderDataset(image_folder_dataset_dir, ... num_parallel_workers=8, ... sampler=sampler)
- Raises
TypeError – If start_index is not an integer value.
TypeError – If num_samples is not an integer value.
RuntimeError – If start_index is a negative value.
RuntimeError – If num_samples is a negative value.
-
class
tinyms.data.SubsetRandomSampler(indices, num_samples=None)[source]¶ Samples the elements randomly from a sequence of indices.
- Parameters
indices (Any iterable python object but string) – A sequence of indices.
num_samples (int, optional) – Number of elements to sample (default=None, all elements).
Examples
>>> indices = [0, 1, 2, 3, 7, 88, 119] >>> >>> # create a SubsetRandomSampler, will sample from the provided indices >>> sampler = ds.SubsetRandomSampler(indices) >>> data = ds.ImageFolderDataset(image_folder_dataset_dir, num_parallel_workers=8, sampler=sampler)
- Raises
TypeError – If type of indices element is not a number.
TypeError – If num_samples is not an integer value.
RuntimeError – If num_samples is a negative value.
-
class
tinyms.data.WeightedRandomSampler(weights, num_samples=None, replacement=True)[source]¶ Samples the elements from [0, len(weights) - 1] randomly with the given weights (probabilities).
- Parameters
Examples
>>> weights = [0.9, 0.01, 0.4, 0.8, 0.1, 0.1, 0.3] >>> >>> # creates a WeightedRandomSampler that will sample 4 elements without replacement >>> sampler = ds.WeightedRandomSampler(weights, 4) >>> dataset = ds.ImageFolderDataset(image_folder_dataset_dir, ... num_parallel_workers=8, ... sampler=sampler)
- Raises
TypeError – If type of weights element is not a number.
TypeError – If num_samples is not an integer value.
TypeError – If replacement is not a boolean value.
RuntimeError – If weights is empty or all zero.
RuntimeError – If num_samples is a negative value.
-
class
tinyms.data.SubsetSampler(indices, num_samples=None)[source]¶ Samples the elements from a sequence of indices.
- Parameters
indices (Any iterable python object but string) – A sequence of indices.
num_samples (int, optional) – Number of elements to sample (default=None, all elements).
Examples
>>> indices = [0, 1, 2, 3, 4, 5] >>> >>> # creates a SubsetSampler, will sample from the provided indices >>> sampler = ds.SubsetSampler(indices) >>> dataset = ds.ImageFolderDataset(image_folder_dataset_dir, ... num_parallel_workers=8, ... sampler=sampler)
- Raises
TypeError – If type of indices element is not a number.
TypeError – If num_samples is not an integer value.
RuntimeError – If num_samples is a negative value.
-
class
tinyms.data.DatasetCache(session_id, size=0, spilling=False, hostname=None, port=None, num_connections=None, prefetch_size=None)[source]¶ A client to interface with tensor caching service.
For details, please check Chinese tutorial, Chinese programming guide.
- Parameters
session_id (int) – A user assigned session id for the current pipeline.
size (int, optional) – Size of the memory set aside for the row caching (default=0 which means unlimited, note that it might bring in the risk of running out of memory on the machine).
spilling (bool, optional) – Whether or not spilling to disk if out of memory (default=False).
hostname (str, optional) – Host name (default=”127.0.0.1”).
port (int, optional) – Port to connect to server (default=50052).
num_connections (int, optional) – Number of tcp/ip connections (default=12).
prefetch_size (int, optional) – Prefetch size (default=20).
Examples
>>> import mindspore.dataset as ds >>> >>> # create a cache instance, in which session_id is generated from command line `cache_admin -g` >>> some_cache = ds.DatasetCache(session_id=session_id, size=0) >>> >>> dataset_dir = "path/to/imagefolder_directory" >>> ds1 = ds.ImageFolderDataset(dataset_dir, cache=some_cache)
-
class
tinyms.data.Schema(schema_file=None)[source]¶ Class to represent a schema of a dataset.
- Parameters
schema_file (str) – Path of schema file (default=None).
- Returns
Schema object, schema info about dataset.
- Raises
RuntimeError – If schema file failed to load.
Example
>>> import mindspore.common.dtype as mstype >>> >>> # Create schema; specify column name, mindspore.dtype and shape of the column >>> schema = ds.Schema() >>> schema.add_column(name='col1', de_type=mstype.int64, shape=[2])
-
add_column(name, de_type, shape=None)[source]¶ Add new column to the schema.
- Parameters
- Raises
ValueError – If column type is unknown.
-
from_json(json_obj)[source]¶ Get schema file from JSON object.
- Parameters
json_obj (dictionary) – Object of JSON parsed.
- Raises
RuntimeError – if there is unknown item in the object.
RuntimeError – if dataset type is missing in the object.
RuntimeError – if columns are missing in the object.
-
parse_columns(columns)[source]¶ Parse the columns and add it to self.
- Parameters
columns (Union[dict, list[dict], tuple[dict]]) –
Dataset attribute information, decoded from schema file.
list[dict], ‘name’ and ‘type’ must be in keys, ‘shape’ optional.
dict, columns.keys() as name, columns.values() is dict, and ‘type’ inside, ‘shape’ optional.
- Raises
RuntimeError – If failed to parse columns.
RuntimeError – If column’s name field is missing.
RuntimeError – If column’s type field is missing.
Example
>>> schema = Schema() >>> columns1 = [{'name': 'image', 'type': 'int8', 'shape': [3, 3]}, >>> {'name': 'label', 'type': 'int8', 'shape': [1]}] >>> schema.parse_columns(columns1) >>> columns2 = {'image': {'shape': [3, 3], 'type': 'int8'}, 'label': {'shape': [1], 'type': 'int8'}} >>> schema.parse_columns(columns2)
-
tinyms.data.zip(datasets)[source]¶ Zip the datasets in the input tuple of datasets.
- Parameters
datasets (tuple of class Dataset) – A tuple of datasets to be zipped together. The number of datasets must be more than 1.
- Returns
ZipDataset, dataset zipped.
- Raises
ValueError – If the number of datasets is 1.
TypeError – If datasets is not a tuple.
Examples
>>> # Create a dataset which is the combination of dataset_1 and dataset_2 >>> dataset = ds.zip((dataset_1, dataset_2))
-
class
tinyms.data.FileWriter(file_name, shard_num=1)[source]¶ Class to write user defined raw data into MindRecord File series.
Note
The mindrecord file may fail to be read if the file name is modified.
- Parameters
- Raises
ParamValueError – If file_name or shard_num is invalid.
-
add_index(index_fields)[source]¶ Select index fields from schema to accelerate reading.
- Parameters
index_fields (list[str]) – Fields would be set as index which should be primitive type.
- Returns
MSRStatus, SUCCESS or FAILED.
- Raises
ParamTypeError – If index field is invalid.
MRMDefineIndexError – If index field is not primitive type.
MRMAddIndexError – If failed to add index field.
MRMGetMetaError – If the schema is not set or failed to get meta.
-
add_schema(content, desc=None)[source]¶ Return a schema id if schema is added successfully, or raise an exception.
- Parameters
- Returns
int, schema id.
- Raises
MRMInvalidSchemaError – If schema is invalid.
MRMBuildSchemaError – If failed to build schema.
MRMAddSchemaError – If failed to add schema.
-
commit()[source]¶ Flush data to disk and generate the corresponding database files.
- Returns
MSRStatus, SUCCESS or FAILED.
- Raises
MRMOpenError – If failed to open MindRecord File.
MRMSetHeaderError – If failed to set header.
MRMIndexGeneratorError – If failed to create index generator.
MRMGenerateIndexError – If failed to write to database.
MRMCommitError – If failed to flush data to disk.
-
classmethod
open_for_append(file_name)[source]¶ Open MindRecord file and get ready to append data.
- Parameters
file_name (str) – String of MindRecord file name.
- Returns
FileWriter, file writer for the opened MindRecord file.
- Raises
ParamValueError – If file_name is invalid.
FileNameError – If path contains invalid characters.
MRMOpenError – If failed to open MindRecord File.
MRMOpenForAppendError – If failed to open file for appending data.
-
set_header_size(header_size)[source]¶ Set the size of header which contains shard information, schema information, page meta information, etc. The larger the header, the more training data a single Mindrecord file can store.
- Parameters
header_size (int) – Size of header, between 16KB and 128MB.
- Returns
MSRStatus, SUCCESS or FAILED.
- Raises
MRMInvalidHeaderSizeError – If failed to set header size.
-
set_page_size(page_size)[source]¶ Set the size of page which mainly refers to the block to store training data, and the training data will be split into raw page and blob page in mindrecord. The larger the page, the more training data a single page can store.
- Parameters
page_size (int) – Size of page, between 32KB and 256MB.
- Returns
MSRStatus, SUCCESS or FAILED.
- Raises
MRMInvalidPageSizeError – If failed to set page size.
-
write_raw_data(raw_data, parallel_writer=False)[source]¶ Write raw data and generate sequential pair of MindRecord File and validate data based on predefined schema by default.
- Parameters
- Returns
MSRStatus, SUCCESS or FAILED.
- Raises
ParamTypeError – If index field is invalid.
MRMOpenError – If failed to open MindRecord File.
MRMValidateDataError – If data does not match blob fields.
MRMSetHeaderError – If failed to set header.
MRMWriteDatasetError – If failed to write dataset.
-
class
tinyms.data.FileReader(file_name, num_consumer=4, columns=None, operator=None)[source]¶ Class to read MindRecord File series.
- Parameters
file_name (str, list[str]) – One of MindRecord File or a file list.
num_consumer (int, optional) – Number of consumer threads which load data to memory (default=4). It should not be smaller than 1 or larger than the number of CPUs.
columns (list[str], optional) – A list of fields where corresponding data would be read (default=None).
operator (int, optional) – Reserved parameter for operators (default=None).
- Raises
ParamValueError – If file_name, num_consumer or columns is invalid.
-
class
tinyms.data.MindPage(file_name, num_consumer=4)[source]¶ Class to read MindRecord File series in pagination.
- Parameters
- Raises
ParamValueError – If file_name, num_consumer or columns is invalid.
MRMInitSegmentError – If failed to initialize ShardSegment.
-
property
candidate_fields¶ Return candidate category fields.
- Returns
list[str], by which data could be grouped.
-
property
category_field¶ Getter function for category fields.
- Returns
list[str], by which data could be grouped.
-
get_category_fields()[source]¶ Return candidate category fields.
- Returns
list[str], by which data could be grouped.
-
read_at_page_by_id(category_id, page, num_row)[source]¶ Query by category id in pagination.
- Parameters
- Returns
list[dict], data queried by category id.
- Raises
ParamValueError – If any parameter is invalid.
MRMFetchDataError – If failed to fetch data by category.
MRMUnsupportedSchemaError – If schema is invalid.
-
class
tinyms.data.Cifar10ToMR(source, destination)[source]¶ A class to transform from cifar10 to MindRecord.
- Parameters
- Raises
ValueError – If source or destination is invalid.
-
class
tinyms.data.Cifar100ToMR(source, destination)[source]¶ A class to transform from cifar100 to MindRecord.
- Parameters
- Raises
ValueError – If source or destination is invalid.
-
class
tinyms.data.CsvToMR(source, destination, columns_list=None, partition_number=1)[source]¶ A class to transform from csv to MindRecord.
- Parameters
- Raises
ValueError – If source, destination, partition_number is invalid.
RuntimeError – If columns_list is invalid.
-
class
tinyms.data.ImageNetToMR(map_file, image_dir, destination, partition_number=1)[source]¶ A class to transform from imagenet to MindRecord.
- Parameters
map_file (str) –
the map file that indicates label. The map file content should be like this:
n02119789 0 n02100735 1 n02110185 2 n02096294 3
image_dir (str) – image directory contains n02119789, n02100735, n02110185 and n02096294 directory.
destination (str) – the MindRecord file path to transform into.
partition_number (int, optional) – partition size (default=1).
- Raises
ValueError – If map_file, image_dir or destination is invalid.
-
class
tinyms.data.MnistToMR(source, destination, partition_number=1)[source]¶ A class to transform from Mnist to MindRecord.
- Parameters
- Raises
ValueError – If source, destination, partition_number is invalid.
-
class
tinyms.data.TFRecordToMR(source, destination, feature_dict, bytes_fields=None)[source]¶ A class to transform from TFRecord to MindRecord.
- Parameters
source (str) – the TFRecord file to be transformed.
destination (str) – the MindRecord file path to transform into.
feature_dict (dict) –
a dictionary that states the feature type, e.g. feature_dict = {“xxxx”: tf.io.FixedLenFeature([], tf.string), “yyyy”: tf.io.FixedLenFeature([], tf.int64)}
Follow case which uses VarLenFeature is not supported.
feature_dict = {“context”: {“xxxx”: tf.io.FixedLenFeature([], tf.string), “yyyy”: tf.io.VarLenFeature(tf.int64)}, “sequence”: {“zzzz”: tf.io.FixedLenSequenceFeature([], tf.float32)}}
bytes_fields (list, optional) – the bytes fields which are in feature_dict and can be images bytes.
- Raises
ValueError – If parameter is invalid.
Exception – when tensorflow module is not found or version is not correct.
-
run()[source]¶ Execute transformation from TFRecord to MindRecord.
- Returns
MSRStatus, whether TFRecord is successfully transformed to MindRecord.
-
tinyms.data.download_dataset(dataset_name, local_path='.')[source]¶ This function is defined to easily download any public dataset without specifing much details.
- Parameters
- Returns
str, the source location of dataset downloaded.
Examples
>>> from tinyms.data import download_dataset >>> >>> ds_path = download_dataset('mnist')
-
tinyms.data.generate_image_list(dir_path, max_dataset_size=inf)[source]¶ Traverse the directory to generate a list of images path.
tinyms.vision¶
This module is to support vision augmentations. transforms is a high performance image augmentation module which is developed with C++ OpenCV.
-
class
tinyms.vision.ImageViewer(image, label=None)[source]¶ ImageViewer is a class defined for visualizing the input image.
- Parameters
image (Union[PIL.Image, numpy.ndarray]) – image input.
label (str, optional) – specifies the label of this image. Default: None.
- Raises
TypeError – When image input is not numpy.ndarray or PIL.Image.
-
draw(pred_res, labels)[source]¶ Draw the predicting boxes on the picture and show the visualized picture.
- Parameters
Note
This function is only valid when being called in interactive environment, such like Jupyter notebook.
Examples
>>> form PIL import Image >>> >>> img = Image.open('example.jpg') >>> img_viewer = ImageViewer(img) >>> labels = ['1', '2', '3'] >>> img_viewer.draw(pred_res, labels)
-
show()[source]¶ Directly show the visualized picture.
Note
This function is only valid when being called in interactive environment, such like Jupyter notebook.
Examples
>>> form PIL import Image >>> >>> img = Image.open('example.jpg') >>> img_viewer = ImageViewer(img, label='cat') >>> img_viewer.show()
-
tinyms.vision.ssd_bboxes_encode(boxes)[source]¶ Labels anchors with ground truth inputs.
- Parameters
boxes (numpy.ndarray) – Ground truth with shape [N, 5], for each row, it stores [ymin, xmin, ymax, xmax, cls].
- Returns
numpy.ndarray, location ground truth with shape [num_anchors, 4]. numpy.ndarray, class ground truth with shape [num_anchors, 1]. numpy.ndarray, number of positives in an image.
-
tinyms.vision.ssd_bboxes_filter(boxes, box_scores, image_shape)[source]¶ Filter predict boxes with minimum score and nms threshold.
- Parameters
boxes (numpy.ndarray) – Ground truth with shape [N, 4], for each row, it stores [ymin, xmin, ymax, xmax].
box_scores (numpy.ndarray) – Class scores with shape [N, 21].
image_shape (tuple) – Shape of original image with the format [h, w].
- Returns
list[list[float]], ground truth with shape [N, 4], for each row, it stores [ymin, xmin, ymax, xmax]. list[list[float]], class scores with shape [N, 21]. list[list[int]], class label with shape [N, 21].
-
class
tinyms.vision.MnistTransform[source]¶ Mnist dataset transform class.
- Inputs:
img (Union[numpy.ndarray, PIL.Image]): Image to be transformed in Mnist-style.
- Outputs:
numpy.ndarray, transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import MnistTransform >>> >>> mnist_transform = MnistTransform() >>> img = Image.open('example.jpg') >>> img = mnist_transform(img)
-
apply_ds(mnist_ds, repeat_size=1, batch_size=32, num_parallel_workers=None)[source]¶ Apply preprocess operation on MnistDataset instance.
- Parameters
mnist_ds (data.MnistDataset) – MnistDataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: None.
- Returns
data.MnistDataset, the preprocessed MnistDataset instance.
Examples
>>> from tinyms.vision import MnistTransform >>> >>> mnist_transform = MnistTransform() >>> mnist_ds = mnist_transform.apply_ds(mnist_ds)
-
postprocess(input, strategy='TOP1_CLASS')¶ Apply postprocess operation for prediction result.
- Parameters
input (numpy.ndarray) – Prediction result.
strategy (str) – Specifies the postprocess strategy. Default: TOP1_CLASS.
- Returns
str, the postprocess result.
-
class
tinyms.vision.Cifar10Transform[source]¶ Cifar10 dataset transform class.
- Inputs:
img (Union[numpy.ndarray, PIL.Image]): Image to be transformed in Cifar10-style.
- Outputs:
numpy.ndarray, Transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import Cifar10Transform >>> >>> cifar10_transform = Cifar10Transform() >>> img = Image.open('example.jpg') >>> img = cifar10_transform(img)
“””
-
apply_ds(cifar10_ds, repeat_size=1, batch_size=32, num_parallel_workers=None, is_training=True)[source]¶ Apply preprocess operation on Cifar10Dataset instance.
- Parameters
cifar10_ds (data.Cifar10Dataset) – Cifar10Dataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: None.
is_training (bool) – Specifies if is in training step. Default: True.
- Returns
data.Cifar10Dataset, the preprocessed Cifar10Dataset instance.
Examples
>>> from tinyms.vision import Cifar10Transform >>> >>> cifar10_transform = Cifar10Transform() >>> cifar10_ds = cifar10_transform.apply_ds(cifar10_ds)
-
postprocess(input, strategy='TOP1_CLASS')¶ Apply postprocess operation for prediction result.
- Parameters
input (numpy.ndarray) – Prediction result.
strategy (str) – Specifies the postprocess strategy. Default: TOP1_CLASS.
- Returns
str, the postprocess result.
-
class
tinyms.vision.ImageFolderTransform[source]¶ ImageFolder dataset transform class.
- Inputs:
img (Union[numpy.ndarray, PIL.Image]): Image to be transformed in ImageFolder-style.
- Outputs:
numpy.ndarray, transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import ImageFolderTransform >>> >>> imagefolder_transform = ImageFolderTransform() >>> img = Image.open('example.jpg') >>> img = imagefolder_transform(img)
-
apply_ds(imagefolder_ds, repeat_size=1, batch_size=32, num_parallel_workers=None, is_training=True)[source]¶ Apply preprocess operation on ImageFolderDataset instance.
- Parameters
cifar10_ds (data.ImageFolderDataset) – ImageFolderDataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: None.
is_training (bool) – Specifies if is in training step. Default: True.
- Returns
data.ImageFolderDataset, the preprocessed ImageFolderDataset instance.
Examples
>>> from tinyms.vision import ImageFolderTransform >>> >>> imagefolder_transform = ImageFolderTransform() >>> imagefolder_ds = imagefolder_transform.apply_ds(imagefolder_ds)
-
postprocess(input, strategy='TOP1_CLASS')¶ Apply postprocess operation for prediction result.
- Parameters
input (numpy.ndarray) – Prediction result.
strategy (str) – Specifies the postprocess strategy. Default: TOP1_CLASS.
- Returns
str, the postprocess result.
-
class
tinyms.vision.VOCTransform[source]¶ VOC dataset transform class.
- Inputs:
img (Union[numpy.ndarray, PIL.Image]): Image to be transformed in VOC-style.
- Outputs:
numpy.ndarray, transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import VOCTransform >>> >>> voc_transform = VOCTransform() >>> img = Image.open('example.jpg') >>> img = voc_transform(img)
-
apply_ds(voc_ds, repeat_size=1, batch_size=32, num_parallel_workers=None, is_training=True)[source]¶ Apply preprocess operation on VOCDataset instance.
- Parameters
cifar10_ds (data.VOCDataset) – VOCDataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: None.
is_training (bool) – Specifies if is in training step. Default: True.
- Returns
data.VOCDataset, the preprocessed VOCDataset instance.
Examples
>>> from tinyms.vision import VOCTransform >>> >>> VOC_transform = VOCTransform() >>> voc_ds = voc_transform.apply_ds(voc_ds)
-
postprocess(input, image_shape, strategy='TOP1_CLASS')[source]¶ Apply postprocess operation for prediction result.
- Parameters
input (numpy.ndarray) – Prediction result.
image_shape (tuple) – Image shape.
strategy (str) – Specifies the postprocess strategy. Default: TOP1_CLASS.
- Returns
dict, the postprocess result.
-
class
tinyms.vision.CycleGanDatasetTransform[source]¶ CycleGan dataset transform class.
- Inputs:
img (Union[numpy.ndarray, PIL.Image]): Image to be transformed in city_scape.
- Outputs:
numpy.ndarray, transformed image.
Examples
>>> from PIL import Image >>> from tinyms.vision import CycleGanDatasetTransform >>> >>> cyclegan_transform = CycleGanDatasetTransform() >>> img = Image.open('example.jpg') >>> img = cyclegan_transform(img)
-
apply_ds(gan_generator_ds, repeat_size=1, batch_size=1, num_parallel_workers=1, shuffle=True, phase='train')[source]¶ Apply preprocess operation on GeneratorDataset instance.
- Parameters
gan_generator_ds (data.GeneratorDataset) – GeneratorDataset instance.
repeat_size (int) – The repeat size of dataset. Default: 1.
batch_size (int) – Batch size. Default: 32.
num_parallel_workers (int) – The number of concurrent workers. Default: 1.
shuffle (bool) – Specifies if applying shuffle operation. Default: True.
phase (str) – Specifies the current phase. Default: train.
- Returns
data.GeneratorDataset, the preprocessed GeneratorDataset instance.
Examples
>>> from tinyms.vision import CycleGanDatasetTransform >>> >>> cyclegan_transform = CycleGanDatasetTransform() >>> gan_generator_ds = cyclegan_transform.apply_ds(gan_generator_ds)
- Raises
TypeError – If gan_generator_ds is not instance of GeneratorDataset.
-
class
tinyms.vision.AutoContrast(cutoff=0.0, ignore=None)[source]¶ Apply automatic contrast on input image.
- Parameters
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.AutoContrast(cutoff=10.0, ignore=[10, 20])] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.BoundingBoxAugment(transform, ratio=0.3)[source]¶ Apply a given image transform on a random selection of bounding box regions of a given image.
- Parameters
transform – C++ transformation function to be applied on random selection of bounding box regions of a given image.
ratio (float, optional) – Ratio of bounding boxes to apply augmentation on. Range: [0, 1] (default=0.3).
Examples
>>> # set bounding box operation with ratio of 1 to apply rotation on all bounding boxes >>> bbox_aug_op = c_vision.BoundingBoxAugment(c_vision.RandomRotation(90), 1) >>> # map to apply ops >>> image_folder_dataset = image_folder_dataset.map(operations=[bbox_aug_op], ... input_columns=["image", "bbox"], ... output_columns=["image", "bbox"], ... column_order=["image", "bbox"])
-
class
tinyms.vision.CenterCrop(size)[source]¶ Crop the input image at the center to the given size.
- Parameters
size (Union[int, sequence]) – The output size of the cropped image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
Examples
>>> # crop image to a square >>> transforms_list1 = [c_vision.Decode(), c_vision.CenterCrop(50)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list1, ... input_columns=["image"]) >>> # crop image to portrait style >>> transforms_list2 = [c_vision.Decode(), c_vision.CenterCrop((60, 40))] >>> image_folder_dataset_1 = image_folder_dataset_1.map(operations=transforms_list2, ... input_columns=["image"])
-
class
tinyms.vision.CutMixBatch(image_batch_format, alpha=1.0, prob=1.0)[source]¶ Apply CutMix transformation on input batch of images and labels. Note that you need to make labels into one-hot format and batch before calling this function.
- Parameters
image_batch_format (Image Batch Format) – The method of padding. Can be any of [ImageBatchFormat.NHWC, ImageBatchFormat.NCHW]
alpha (float, optional) – hyperparameter of beta distribution (default = 1.0).
prob (float, optional) – The probability by which CutMix is applied to each image (default = 1.0).
Examples
>>> from mindspore.dataset.vision import ImageBatchFormat >>> onehot_op = c_transforms.OneHot(num_classes=10) >>> image_folder_dataset= image_folder_dataset.map(operations=onehot_op, ... input_columns=["label"]) >>> cutmix_batch_op = c_vision.CutMixBatch(ImageBatchFormat.NHWC, 1.0, 0.5) >>> image_folder_dataset = image_folder_dataset.batch(5) >>> image_folder_dataset = image_folder_dataset.map(operations=cutmix_batch_op, ... input_columns=["image", "label"])
-
class
tinyms.vision.CutOut(length, num_patches=1)[source]¶ Randomly cut (mask) out a given number of square patches from the input NumPy image array.
- Parameters
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.CutOut(80, num_patches=10)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.Decode(rgb=True)[source]¶ Decode the input image in RGB mode.
- Parameters
rgb (bool, optional) – Mode of decoding input image (default=True). If True means format of decoded image is RGB else BGR(deprecated).
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.RandomHorizontalFlip()] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.Equalize[source]¶ Apply histogram equalization on input image.
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.Equalize()] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.Grayscale(num_output_channels=1)[source]¶ Convert the input PIL image to grayscale image.
- Parameters
num_output_channels (int) – Number of channels of the output grayscale image (1 or 3). Default is 1. If set to 3, the returned image has 3 identical RGB channels.
Examples
>>> from mindspore.dataset.transforms.py_transforms import Compose >>> transforms_list = Compose([py_vision.Decode(), ... py_vision.Grayscale(3), ... py_vision.ToTensor()]) >>> # apply the transform to dataset through map function >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns="image")
-
class
tinyms.vision.HWC2CHW[source]¶ Transpose the input image; shape (H, W, C) to shape (C, H, W).
Examples
>>> transforms_list = [c_vision.Decode(), ... c_vision.RandomHorizontalFlip(0.75), ... c_vision.RandomCrop(512), ... c_vision.HWC2CHW()] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.Invert[source]¶ Apply invert on input image in RGB mode.
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.Invert()] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.MixUpBatch(alpha=1.0)[source]¶ Apply MixUp transformation on input batch of images and labels. Each image is multiplied by a random weight (lambda) and then added to a randomly selected image from the batch multiplied by (1 - lambda). The same formula is also applied to the one-hot labels. Note that you need to make labels into one-hot format and batch before calling this function.
- Parameters
alpha (float, optional) – Hyperparameter of beta distribution (default = 1.0).
Examples
>>> onehot_op = c_transforms.OneHot(num_classes=10) >>> image_folder_dataset= image_folder_dataset.map(operations=onehot_op, ... input_columns=["label"]) >>> mixup_batch_op = c_vision.MixUpBatch(alpha=0.9) >>> image_folder_dataset = image_folder_dataset.batch(5) >>> image_folder_dataset = image_folder_dataset.map(operations=mixup_batch_op, ... input_columns=["image", "label"])
-
class
tinyms.vision.Normalize(mean, std)[source]¶ Normalize the input image with respect to mean and standard deviation.
- Parameters
mean (sequence) – List or tuple of mean values for each channel, with respect to channel order. The mean values must be in range [0.0, 255.0].
std (sequence) – List or tuple of standard deviations for each channel, with respect to channel order. The standard deviation values must be in range (0.0, 255.0].
Examples
>>> decode_op = c_vision.Decode() >>> normalize_op = c_vision.Normalize(mean=[121.0, 115.0, 100.0], std=[70.0, 68.0, 71.0]) >>> transforms_list = [decode_op, normalize_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.Pad(padding, fill_value=0, padding_mode=<Border.CONSTANT: 'constant'>)[source]¶ Pad the image according to padding parameters.
- Parameters
padding (Union[int, sequence]) – The number of pixels to pad the image. If a single number is provided, it pads all borders with this value. If a tuple or list of 2 values are provided, it pads the (left and top) with the first value and (right and bottom) with the second value. If 4 values are provided as a list or tuple, it pads the left, top, right and bottom respectively.
fill_value (Union[int, tuple], optional) – The pixel intensity of the borders, only valid for padding_mode Border.CONSTANT. If it is a 3-tuple, it is used to fill R, G, B channels respectively. If it is an integer, it is used for all RGB channels. The fill_value values must be in range [0, 255] (default=0).
padding_mode (Border mode, optional) –
The method of padding (default=Border.CONSTANT). Can be any of [Border.CONSTANT, Border.EDGE, Border.REFLECT, Border.SYMMETRIC].
Border.CONSTANT, means it fills the border with constant values.
Border.EDGE, means it pads with the last value on the edge.
Border.REFLECT, means it reflects the values on the edge omitting the last value of edge.
Border.SYMMETRIC, means it reflects the values on the edge repeating the last value of edge.
Examples
>>> from mindspore.dataset.vision import Border >>> transforms_list = [c_vision.Decode(), c_vision.Pad([100, 100, 100, 100])] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
tinyms.vision.PILRandomHorizontalFlip¶ alias of
mindspore.dataset.vision.py_transforms.RandomHorizontalFlip
-
class
tinyms.vision.RandomAffine(degrees, translate=None, scale=None, shear=None, resample=<Inter.NEAREST: 0>, fill_value=0)[source]¶ Apply Random affine transformation to the input image.
- Parameters
degrees (int or float or sequence) – Range of the rotation degrees. If degrees is a number, the range will be (-degrees, degrees). If degrees is a sequence, it should be (min, max).
translate (sequence, optional) – Sequence (tx_min, tx_max, ty_min, ty_max) of minimum/maximum translation in x(horizontal) and y(vertical) directions (default=None). The horizontal and vertical shift is selected randomly from the range: (tx_min*width, tx_max*width) and (ty_min*height, ty_max*height), respectively. If a tuple or list of size 2, then a translate parallel to the X axis in the range of (translate[0], translate[1]) is applied. If a tuple of list of size 4, then a translate parallel to the X axis in the range of (translate[0], translate[1]) and a translate parallel to the Y axis in the range of (translate[2], translate[3]) are applied. If None, no translation is applied.
scale (sequence, optional) – Scaling factor interval (default=None, original scale is used).
shear (int or float or sequence, optional) – Range of shear factor (default=None). If a number, then a shear parallel to the X axis in the range of (-shear, +shear) is applied. If a tuple or list of size 2, then a shear parallel to the X axis in the range of (shear[0], shear[1]) is applied. If a tuple of list of size 4, then a shear parallel to X axis in the range of (shear[0], shear[1]) and a shear parallel to Y axis in the range of (shear[2], shear[3]) is applied. If None, no shear is applied.
resample (Inter mode, optional) –
An optional resampling filter (default=Inter.NEAREST). If omitted, or if the image has mode “1” or “P”, it is set to be Inter.NEAREST. It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means resample method is bilinear interpolation.
Inter.NEAREST, means resample method is nearest-neighbor interpolation.
Inter.BICUBIC, means resample method is bicubic interpolation.
fill_value (tuple or int, optional) – Optional fill_value to fill the area outside the transform in the output image. There must be three elements in tuple and the value of single element is [0, 255]. Used only in Pillow versions > 5.0.0 (default=0, filling is performed).
- Raises
ValueError – If degrees is negative.
ValueError – If translation value is not between -1 and 1.
ValueError – If scale is not positive.
ValueError – If shear is a number but is not positive.
TypeError – If degrees is not a number or a list or a tuple. If degrees is a list or tuple, its length is not 2.
TypeError – If translate is specified but is not list or a tuple of length 2 or 4.
TypeError – If scale is not a list or tuple of length 2.’’
TypeError – If shear is not a list or tuple of length 2 or 4.
TypeError – If fill_value is not a single integer or a 3-tuple.
Examples
>>> from mindspore.dataset.vision import Inter >>> decode_op = c_vision.Decode() >>> random_affine_op = c_vision.RandomAffine(degrees=15, ... translate=(-0.1, 0.1, 0, 0), ... scale=(0.9, 1.1), ... resample=Inter.NEAREST) >>> transforms_list = [decode_op, random_affine_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomColor(degrees=(0.1, 1.9))[source]¶ Adjust the color of the input image by a fixed or random degree. This operation works only with 3-channel color images.
- Parameters
degrees (sequence, optional) – Range of random color adjustment degrees. It should be in (min, max) format. If min=max, then it is a single fixed magnitude operation (default=(0.1, 1.9)).
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.RandomColor((0.5, 2.0))] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomColorAdjust(brightness=(1, 1), contrast=(1, 1), saturation=(1, 1), hue=(0, 0))[source]¶ Randomly adjust the brightness, contrast, saturation, and hue of the input image.
- Parameters
brightness (Union[float, list, tuple], optional) – Brightness adjustment factor (default=(1, 1)). Cannot be negative. If it is a float, the factor is uniformly chosen from the range [max(0, 1-brightness), 1+brightness]. If it is a sequence, it should be [min, max] for the range.
contrast (Union[float, list, tuple], optional) – Contrast adjustment factor (default=(1, 1)). Cannot be negative. If it is a float, the factor is uniformly chosen from the range [max(0, 1-contrast), 1+contrast]. If it is a sequence, it should be [min, max] for the range.
saturation (Union[float, list, tuple], optional) – Saturation adjustment factor (default=(1, 1)). Cannot be negative. If it is a float, the factor is uniformly chosen from the range [max(0, 1-saturation), 1+saturation]. If it is a sequence, it should be [min, max] for the range.
hue (Union[float, list, tuple], optional) – Hue adjustment factor (default=(0, 0)). If it is a float, the range will be [-hue, hue]. Value should be 0 <= hue <= 0.5. If it is a sequence, it should be [min, max] where -0.5 <= min <= max <= 0.5.
Examples
>>> decode_op = c_vision.Decode() >>> transform_op = c_vision.RandomColorAdjust(brightness=(0.5, 1), ... contrast=(0.4, 1), ... saturation=(0.3, 1)) >>> transforms_list = [decode_op, transform_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomCrop(size, padding=None, pad_if_needed=False, fill_value=0, padding_mode=<Border.CONSTANT: 'constant'>)[source]¶ Crop the input image at a random location.
- Parameters
size (Union[int, sequence]) – The output size of the cropped image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
padding (Union[int, sequence], optional) – The number of pixels to pad the image (default=None). If padding is not None, pad image firstly with padding values. If a single number is provided, pad all borders with this value. If a tuple or list of 2 values are provided, pad the (left and top) with the first value and (right and bottom) with the second value. If 4 values are provided as a list or tuple, pad the left, top, right and bottom respectively.
pad_if_needed (bool, optional) – Pad the image if either side is smaller than the given output size (default=False).
fill_value (Union[int, tuple], optional) – The pixel intensity of the borders, only valid for padding_mode Border.CONSTANT. If it is a 3-tuple, it is used to fill R, G, B channels respectively. If it is an integer, it is used for all RGB channels. The fill_value values must be in range [0, 255] (default=0).
padding_mode (Border mode, optional) –
The method of padding (default=Border.CONSTANT). It can be any of [Border.CONSTANT, Border.EDGE, Border.REFLECT, Border.SYMMETRIC].
Border.CONSTANT, means it fills the border with constant values.
Border.EDGE, means it pads with the last value on the edge.
Border.REFLECT, means it reflects the values on the edge omitting the last value of edge.
Border.SYMMETRIC, means it reflects the values on the edge repeating the last value of edge.
Examples
>>> from mindspore.dataset.vision import Border >>> decode_op = c_vision.Decode() >>> random_crop_op = c_vision.RandomCrop(512, [200, 200, 200, 200], padding_mode=Border.EDGE) >>> transforms_list = [decode_op, random_crop_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomCropDecodeResize(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), interpolation=<Inter.BILINEAR: 2>, max_attempts=10)[source]¶ A combination of Crop, Decode and Resize. It will get better performance for JPEG images.
- Parameters
size (Union[int, sequence]) – The size of the output image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
scale (tuple, optional) – Range [min, max) of respective size of the original size to be cropped (default=(0.08, 1.0)).
ratio (tuple, optional) – Range [min, max) of aspect ratio to be cropped (default=(3. / 4., 4. / 3.)).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.BILINEAR). It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
max_attempts (int, optional) – The maximum number of attempts to propose a valid crop_area (default=10). If exceeded, fall back to use center_crop instead.
Examples
>>> from mindspore.dataset.vision import Inter >>> resize_crop_decode_op = c_vision.RandomCropDecodeResize(size=(50, 75), ... scale=(0.25, 0.5), ... interpolation=Inter.NEAREST, ... max_attempts=5) >>> transforms_list = [resize_crop_decode_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomCropWithBBox(size, padding=None, pad_if_needed=False, fill_value=0, padding_mode=<Border.CONSTANT: 'constant'>)[source]¶ Crop the input image at a random location and adjust bounding boxes accordingly.
- Parameters
size (Union[int, sequence]) – The output size of the cropped image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
padding (Union[int, sequence], optional) – The number of pixels to pad the image (default=None). If padding is not None, first pad image with padding values. If a single number is provided, pad all borders with this value. If a tuple or list of 2 values are provided, pad the (left and top) with the first value and (right and bottom) with the second value. If 4 values are provided as a list or tuple, pad the left, top, right and bottom respectively.
pad_if_needed (bool, optional) – Pad the image if either side is smaller than the given output size (default=False).
fill_value (Union[int, tuple], optional) – The pixel intensity of the borders, only valid for padding_mode Border.CONSTANT. If it is a 3-tuple, it is used to fill R, G, B channels respectively. If it is an integer, it is used for all RGB channels. The fill_value values must be in range [0, 255] (default=0).
padding_mode (Border mode, optional) –
The method of padding (default=Border.CONSTANT). It can be any of [Border.CONSTANT, Border.EDGE, Border.REFLECT, Border.SYMMETRIC].
Border.CONSTANT, means it fills the border with constant values.
Border.EDGE, means it pads with the last value on the edge.
Border.REFLECT, means it reflects the values on the edge omitting the last value of edge.
Border.SYMMETRIC, means it reflects the values on the edge repeating the last value of edge.
Examples
>>> decode_op = c_vision.Decode() >>> random_crop_with_bbox_op = c_vision.RandomCropWithBBox([512, 512], [200, 200, 200, 200]) >>> transforms_list = [decode_op, random_crop_with_bbox_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomHorizontalFlip(prob=0.5)[source]¶ Randomly flip the input image horizontally with a given probability.
- Parameters
prob (float, optional) – Probability of the image being flipped (default=0.5).
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.RandomHorizontalFlip(0.75)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomHorizontalFlipWithBBox(prob=0.5)[source]¶ Flip the input image horizontally, randomly with a given probability and adjust bounding boxes accordingly.
- Parameters
prob (float, optional) – Probability of the image being flipped (default=0.5).
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.RandomHorizontalFlipWithBBox(0.70)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomPosterize(bits=(8, 8))[source]¶ Reduce the number of bits for each color channel.
- Parameters
bits (sequence or int, optional) – Range of random posterize to compress image. Bits values must be in range of [1,8], and include at least one integer value in the given range. It must be in (min, max) or integer format. If min=max, then it is a single fixed magnitude operation (default=(8, 8)).
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.RandomPosterize((6, 8))] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomResize(size)[source]¶ Tensor operation to resize the input image using a randomly selected interpolation mode.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
Examples
>>> # randomly resize image, keeping aspect ratio >>> transforms_list1 = [c_vision.Decode(), c_vision.RandomResize(50)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list1, ... input_columns=["image"]) >>> # randomly resize image to landscape style >>> transforms_list2 = [c_vision.Decode(), c_vision.RandomResize((40, 60))] >>> image_folder_dataset_1 = image_folder_dataset_1.map(operations=transforms_list2, ... input_columns=["image"])
-
class
tinyms.vision.RandomResizedCrop(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), interpolation=<Inter.BILINEAR: 2>, max_attempts=10)[source]¶ Crop the input image to a random size and aspect ratio.
- Parameters
size (Union[int, sequence]) – The size of the output image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
scale (tuple, optional) – Range [min, max) of respective size of the original size to be cropped (default=(0.08, 1.0)).
ratio (tuple, optional) – Range [min, max) of aspect ratio to be cropped (default=(3. / 4., 4. / 3.)).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.BILINEAR). It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
max_attempts (int, optional) – The maximum number of attempts to propose a valid crop_area (default=10). If exceeded, fall back to use center_crop instead.
Examples
>>> from mindspore.dataset.vision import Inter >>> decode_op = c_vision.Decode() >>> resize_crop_op = c_vision.RandomResizedCrop(size=(50, 75), scale=(0.25, 0.5), ... interpolation=Inter.BILINEAR) >>> transforms_list = [decode_op, resize_crop_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomResizedCropWithBBox(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), interpolation=<Inter.BILINEAR: 2>, max_attempts=10)[source]¶ Crop the input image to a random size and aspect ratio and adjust bounding boxes accordingly.
- Parameters
size (Union[int, sequence]) – The size of the output image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
scale (tuple, optional) – Range (min, max) of respective size of the original size to be cropped (default=(0.08, 1.0)).
ratio (tuple, optional) – Range (min, max) of aspect ratio to be cropped (default=(3. / 4., 4. / 3.)).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.BILINEAR). It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
max_attempts (int, optional) – The maximum number of attempts to propose a valid crop area (default=10). If exceeded, fall back to use center crop instead.
Examples
>>> from mindspore.dataset.vision import Inter >>> decode_op = c_vision.Decode() >>> bbox_op = c_vision.RandomResizedCropWithBBox(size=50, interpolation=Inter.NEAREST) >>> transforms_list = [decode_op, bbox_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomResizeWithBBox(size)[source]¶ Tensor operation to resize the input image using a randomly selected interpolation mode and adjust bounding boxes accordingly.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
Examples
>>> # randomly resize image with bounding boxes, keeping aspect ratio >>> transforms_list1 = [c_vision.Decode(), c_vision.RandomResizeWithBBox(60)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list1, ... input_columns=["image"]) >>> # randomly resize image with bounding boxes to portrait style >>> transforms_list2 = [c_vision.Decode(), c_vision.RandomResizeWithBBox((80, 60))] >>> image_folder_dataset_1 = image_folder_dataset_1.map(operations=transforms_list2, ... input_columns=["image"])
-
class
tinyms.vision.RandomRotation(degrees, resample=<Inter.NEAREST: 0>, expand=False, center=None, fill_value=0)[source]¶ Rotate the input image by a random angle.
- Parameters
degrees (Union[int, float, sequence) – Range of random rotation degrees. If degrees is a number, the range will be converted to (-degrees, degrees). If degrees is a sequence, it should be (min, max).
resample (Inter mode, optional) –
An optional resampling filter (default=Inter.NEAREST). If omitted, or if the image has mode “1” or “P”, it is set to be Inter.NEAREST. It can be any of [Inter.BILINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.BILINEAR, means resample method is bilinear interpolation.
Inter.NEAREST, means resample method is nearest-neighbor interpolation.
Inter.BICUBIC, means resample method is bicubic interpolation.
expand (bool, optional) – Optional expansion flag (default=False). If set to True, expand the output image to make it large enough to hold the entire rotated image. If set to False or omitted, make the output image the same size as the input. Note that the expand flag assumes rotation around the center and no translation.
center (tuple, optional) – Optional center of rotation (a 2-tuple) (default=None). Origin is the top left corner. None sets to the center of the image.
fill_value (Union[int, tuple], optional) – Optional fill color for the area outside the rotated image. If it is a 3-tuple, it is used to fill R, G, B channels respectively. If it is an integer, it is used for all RGB channels. The fill_value values must be in range [0, 255] (default=0).
Examples
>>> from mindspore.dataset.vision import Inter >>> transforms_list = [c_vision.Decode(), ... c_vision.RandomRotation(degrees=5.0, ... resample=Inter.NEAREST, ... expand=True)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomSelectSubpolicy(policy)[source]¶ Choose a random sub-policy from a list to be applied on the input image. A sub-policy is a list of tuples (op, prob), where op is a TensorOp operation and prob is the probability that this op will be applied. Once a sub-policy is selected, each op within the subpolicy with be applied in sequence according to its probability.
Examples
>>> policy = [[(c_vision.RandomRotation((45, 45)), 0.5), ... (c_vision.RandomVerticalFlip(), 1), ... (c_vision.RandomColorAdjust(), 0.8)], ... [(c_vision.RandomRotation((90, 90)), 1), ... (c_vision.RandomColorAdjust(), 0.2)]] >>> image_folder_dataset = image_folder_dataset.map(operations=c_vision.RandomSelectSubpolicy(policy), ... input_columns=["image"])
-
class
tinyms.vision.RandomSharpness(degrees=(0.1, 1.9))[source]¶ Adjust the sharpness of the input image by a fixed or random degree. Degree of 0.0 gives a blurred image, degree of 1.0 gives the original image, and degree of 2.0 gives a sharpened image.
- Parameters
degrees (Union[list, tuple], optional) – Range of random sharpness adjustment degrees. It should be in (min, max) format. If min=max, then it is a single fixed magnitude operation (default = (0.1, 1.9)).
- Raises
TypeError – If degrees is not a list or tuple.
ValueError – If degrees is negative.
ValueError – If degrees is in (max, min) format instead of (min, max).
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.RandomSharpness(degrees=(0.2, 1.9))] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomSolarize(threshold=(0, 255))[source]¶ Randomly invert the pixel values of input image within given range.
- Parameters
threshold (tuple, optional) – Range of random solarize threshold (default=(0, 255)). Threshold values should always be in (min, max) format, where min <= max, min and max are integers in the range (0, 255). If min=max, then invert all pixel values above min(max).
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.RandomSolarize(threshold=(10,100))] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomVerticalFlip(prob=0.5)[source]¶ Randomly flip the input image vertically with a given probability.
- Parameters
prob (float, optional) – Probability of the image being flipped (default=0.5).
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.RandomVerticalFlip(0.25)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.RandomVerticalFlipWithBBox(prob=0.5)[source]¶ Flip the input image vertically, randomly with a given probability and adjust bounding boxes accordingly.
- Parameters
prob (float, optional) – Probability of the image being flipped (default=0.5).
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.RandomVerticalFlipWithBBox(0.20)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.Rescale(rescale, shift)[source]¶ Tensor operation to rescale the input image.
Examples
>>> transforms_list = [c_vision.Decode(), c_vision.Rescale(1.0 / 255.0, -1.0)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.Resize(size, interpolation=<Inter.BILINEAR: 2>)[source]¶ Resize the input image to the given size.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, the smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.LINEAR). It can be any of [Inter.LINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.LINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
Inter.AREA, means interpolation method is pixel area interpolation.
Examples
>>> from mindspore.dataset.vision import Inter >>> decode_op = c_vision.Decode() >>> resize_op = c_vision.Resize([100, 75], Inter.BICUBIC) >>> transforms_list = [decode_op, resize_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.ResizeWithBBox(size, interpolation=<Inter.BILINEAR: 2>)[source]¶ Resize the input image to the given size and adjust bounding boxes accordingly.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
interpolation (Inter mode, optional) –
Image interpolation mode (default=Inter.LINEAR). It can be any of [Inter.LINEAR, Inter.NEAREST, Inter.BICUBIC].
Inter.LINEAR, means interpolation method is bilinear interpolation.
Inter.NEAREST, means interpolation method is nearest-neighbor interpolation.
Inter.BICUBIC, means interpolation method is bicubic interpolation.
Examples
>>> from mindspore.dataset.vision import Inter >>> decode_op = c_vision.Decode() >>> bbox_op = c_vision.ResizeWithBBox(50, Inter.NEAREST) >>> transforms_list = [decode_op, bbox_op] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list, ... input_columns=["image"])
-
class
tinyms.vision.SoftDvppDecodeRandomCropResizeJpeg(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), max_attempts=10)[source]¶ Tensor operation to decode, random crop and resize JPEG image using the simulation algorithm of Ascend series chip DVPP module.
The usage scenario is consistent with SoftDvppDecodeResizeJpeg. The input image size should be in range [32*32, 8192*8192]. The zoom-out and zoom-in multiples of the image length and width should in the range [1/32, 16]. Only images with an even resolution can be output. The output of odd resolution is not supported.
- Parameters
size (Union[int, sequence]) – The size of the output image. If size is an integer, a square crop of size (size, size) is returned. If size is a sequence of length 2, it should be (height, width).
scale (tuple, optional) – Range [min, max) of respective size of the original size to be cropped (default=(0.08, 1.0)).
ratio (tuple, optional) – Range [min, max) of aspect ratio to be cropped (default=(3. / 4., 4. / 3.)).
max_attempts (int, optional) – The maximum number of attempts to propose a valid crop_area (default=10). If exceeded, fall back to use center_crop instead.
Examples
>>> # decode, randomly crop and resize image, keeping aspect ratio >>> transforms_list1 = [c_vision.Decode(), c_vision.SoftDvppDecodeRandomCropResizeJpeg(90)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list1, ... input_columns=["image"]) >>> # decode, randomly crop and resize to landscape style >>> transforms_list2 = [c_vision.Decode(), c_vision.SoftDvppDecodeRandomCropResizeJpeg((80, 100))] >>> image_folder_dataset_1 = image_folder_dataset_1.map(operations=transforms_list2, ... input_columns=["image"])
-
class
tinyms.vision.SoftDvppDecodeResizeJpeg(size)[source]¶ Tensor operation to decode and resize JPEG image using the simulation algorithm of Ascend series chip DVPP module.
It is recommended to use this algorithm in the following scenarios: When training, the DVPP of the Ascend chip is not used, and the DVPP of the Ascend chip is used during inference, and the accuracy of inference is lower than the accuracy of training; and the input image size should be in range [32*32, 8192*8192]. The zoom-out and zoom-in multiples of the image length and width should in the range [1/32, 16]. Only images with an even resolution can be output. The output of odd resolution is not supported.
- Parameters
size (Union[int, sequence]) – The output size of the resized image. If size is an integer, smaller edge of the image will be resized to this value with the same image aspect ratio. If size is a sequence of length 2, it should be (height, width).
Examples
>>> # decode and resize image, keeping aspect ratio >>> transforms_list1 = [c_vision.Decode(), c_vision.SoftDvppDecodeResizeJpeg(70)] >>> image_folder_dataset = image_folder_dataset.map(operations=transforms_list1, ... input_columns=["image"]) >>> # decode and resize to portrait style >>> transforms_list2 = [c_vision.Decode(), c_vision.SoftDvppDecodeResizeJpeg((80, 60))] >>> image_folder_dataset_1 = image_folder_dataset_1.map(operations=transforms_list2, ... input_columns=["image"])
-
class
tinyms.vision.UniformAugment(transforms, num_ops=2)[source]¶ Tensor operation to perform randomly selected augmentation.
- Parameters
transforms – List of C++ operations (Python operations are not accepted).
num_ops (int, optional) – Number of operations to be selected and applied (default=2).
Examples
>>> import mindspore.dataset.vision.py_transforms as py_vision >>> transforms_list = [c_vision.RandomHorizontalFlip(), ... c_vision.RandomVerticalFlip(), ... c_vision.RandomColorAdjust(), ... c_vision.RandomRotation(degrees=45)] >>> uni_aug_op = c_vision.UniformAugment(transforms=transforms_list, num_ops=2) >>> transforms_all = [c_vision.Decode(), c_vision.Resize(size=[224, 224]), ... uni_aug_op, py_vision.ToTensor()] >>> image_folder_dataset_1 = image_folder_dataset.map(operations=transforms_all, ... input_columns="image", ... num_parallel_workers=1)
-
class
tinyms.vision.Compose(transforms)[source]¶ Compose a list of transforms into a single transform.
- Parameters
transforms (list) – List of transformations to be applied.
Examples
>>> compose = c_transforms.Compose([c_vision.Decode(), c_vision.RandomCrop(512)]) >>> image_folder_dataset = image_folder_dataset.map(operations=compose)
-
class
tinyms.vision.Concatenate(axis=0, prepend=None, append=None)[source]¶ Tensor operation that concatenates all columns into a single tensor.
- Parameters
axis (int, optional) – Concatenate the tensors along given axis (Default=0).
prepend (numpy.array, optional) – NumPy array to be prepended to the already concatenated tensors (Default=None).
append (numpy.array, optional) – NumPy array to be appended to the already concatenated tensors (Default=None).
Examples
>>> import numpy as np >>> # concatenate string >>> prepend_tensor = np.array(["dw", "df"], dtype='S') >>> append_tensor = np.array(["dwsdf", "df"], dtype='S') >>> concatenate_op = c_transforms.Concatenate(0, prepend_tensor, append_tensor) >>> data = [["This","is","a","string"]] >>> dataset = ds.NumpySlicesDataset(data) >>> dataset = dataset.map(operations=concatenate_op)
-
class
tinyms.vision.Duplicate[source]¶ Duplicate the input tensor to output, only support transform one column each time.
Examples
>>> # Data before >>> # | x | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------+ >>> data = [[1,2,3]] >>> numpy_slices_dataset = ds.NumpySlicesDataset(data, ["x"]) >>> numpy_slices_dataset = numpy_slices_dataset.map(operations=c_transforms.Duplicate(), ... input_columns=["x"], ... output_columns=["x", "y"], ... column_order=["x", "y"]) >>> # Data after >>> # | x | y | >>> # +---------+---------+ >>> # | [1,2,3] | [1,2,3] | >>> # +---------+---------+
-
class
tinyms.vision.Fill(fill_value)[source]¶ Tensor operation to fill all elements in the tensor with the specified value. The output tensor will have the same shape and type as the input tensor.
- Parameters
fill_value (Union[str, bytes, int, float, bool])) – scalar value to fill the tensor with.
Examples
>>> import numpy as np >>> # generate a 1D integer numpy array from 0 to 4 >>> def generator_1d(): ... for i in range(5): ... yield (np.array([i]),) >>> generator_dataset = ds.GeneratorDataset(generator_1d, column_names="col1") >>> # [[0], [1], [2], [3], [4]] >>> fill_op = c_transforms.Fill(3) >>> generator_dataset = generator_dataset.map(operations=fill_op) >>> # [[3], [3], [3], [3], [3]]
-
class
tinyms.vision.Mask(operator, constant, dtype=mindspore.bool)[source]¶ Mask content of the input tensor with the given predicate. Any element of the tensor that matches the predicate will be evaluated to True, otherwise False.
- Parameters
Examples
>>> from mindspore.dataset.transforms.c_transforms import Relational >>> # Data before >>> # | col | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------+ >>> data = [[1, 2, 3]] >>> numpy_slices_dataset = ds.NumpySlicesDataset(data, ["col"]) >>> numpy_slices_dataset = numpy_slices_dataset.map(operations=c_transforms.Mask(Relational.EQ, 2)) >>> # Data after >>> # | col | >>> # +--------------------+ >>> # | [False,True,False] | >>> # +--------------------+
-
class
tinyms.vision.OneHot(num_classes)[source]¶ Tensor operation to apply one hot encoding.
- Parameters
num_classes (int) – Number of classes of objects in dataset. It should be larger than the largest label number in the dataset.
- Raises
RuntimeError – feature size is bigger than num_classes.
Examples
>>> # Assume that dataset has 10 classes, thus the label ranges from 0 to 9 >>> onehot_op = c_transforms.OneHot(num_classes=10) >>> mnist_dataset = mnist_dataset.map(operations=onehot_op, input_columns=["label"])
-
class
tinyms.vision.PadEnd(pad_shape, pad_value=None)[source]¶ Pad input tensor according to pad_shape, need to have same rank.
- Parameters
pad_shape (list(int)) – List of integers representing the shape needed. Dimensions that set to None will not be padded (i.e., original dim will be used). Shorter dimensions will truncate the values.
pad_value (Union[str, bytes, int, float, bool]), optional) – Value used to pad. Default to 0 or empty string in case of tensors of strings.
Examples
>>> # Data before >>> # | col | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------| >>> data = [[1, 2, 3]] >>> numpy_slices_dataset = ds.NumpySlicesDataset(data, ["col"]) >>> numpy_slices_dataset = numpy_slices_dataset.map(operations=c_transforms.PadEnd(pad_shape=[4], ... pad_value=10)) >>> # Data after >>> # | col | >>> # +------------+ >>> # | [1,2,3,10] | >>> # +------------|
-
class
tinyms.vision.RandomApply(transforms, prob=0.5)[source]¶ Randomly perform a series of transforms with a given probability.
- Parameters
Examples
>>> rand_apply = c_transforms.RandomApply([c_vision.RandomCrop(512)]) >>> image_folder_dataset = image_folder_dataset.map(operations=rand_apply)
-
class
tinyms.vision.RandomChoice(transforms)[source]¶ Randomly select one transform from a list of transforms to perform operation.
- Parameters
transforms (list) – List of transformations to be chosen from to apply.
Examples
>>> rand_choice = c_transforms.RandomChoice([c_vision.CenterCrop(50), c_vision.RandomCrop(512)]) >>> image_folder_dataset = image_folder_dataset.map(operations=rand_choice)
-
class
tinyms.vision.Slice(*slices)[source]¶ Slice operation to extract a tensor out using the given n slices.
The functionality of Slice is similar to NumPy’s indexing feature. (Currently only rank-1 tensors are supported).
- Parameters
slices (Union[int, list[int], slice, None, Ellipsis]) –
Maximum n number of arguments to slice a tensor of rank n . One object in slices can be one of:
int: Slice this index only along the first dimension. Negative index is supported.list(int): Slice these indices along the first dimension. Negative indices are supported.slice: Slice the generated indices from the slice object along the first dimension. Similar to start:stop:step.None: Slice the whole dimension. Similar to[:]in Python indexing.Ellipsis: Slice the whole dimension, same result with None.
Examples
>>> # Data before >>> # | col | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------| >>> data = [[1, 2, 3]] >>> numpy_slices_dataset = ds.NumpySlicesDataset(data, ["col"]) >>> # slice indices 1 and 2 only >>> numpy_slices_dataset = numpy_slices_dataset.map(operations=c_transforms.Slice(slice(1,3))) >>> # Data after >>> # | col | >>> # +---------+ >>> # | [2,3] | >>> # +---------|
-
class
tinyms.vision.TypeCast(data_type)[source]¶ Tensor operation to cast to a given MindSpore data type.
- Parameters
data_type (mindspore.dtype) – mindspore.dtype to be cast to.
Examples
>>> import numpy as np >>> import mindspore.common.dtype as mstype >>> >>> # Generate 1d int numpy array from 0 - 63 >>> def generator_1d(): ... for i in range(64): ... yield (np.array([i]),) >>> >>> dataset = ds.GeneratorDataset(generator_1d, column_names='col') >>> type_cast_op = c_transforms.TypeCast(mstype.int32) >>> dataset = dataset.map(operations=type_cast_op)
-
class
tinyms.vision.Unique[source]¶ Perform the unique operation on the input tensor, only support transform one column each time.
Return 3 tensor: unique output tensor, index tensor, count tensor.
Unique output tensor contains all the unique elements of the input tensor in the same order that they occur in the input tensor.
Index tensor that contains the index of each element of the input tensor in the unique output tensor.
Count tensor that contains the count of each element of the output tensor in the input tensor.
Note
Call batch op before calling this function.
Examples
>>> # Data before >>> # | x | >>> # +--------------------+ >>> # | [[0,1,2], [1,2,3]] | >>> # +--------------------+ >>> data = [[[0,1,2], [1,2,3]]] >>> dataset = ds.NumpySlicesDataset(data, ["x"]) >>> dataset = dataset.map(operations=c_transforms.Unique(), ... input_columns=["x"], ... output_columns=["x", "y", "z"], ... column_order=["x", "y", "z"]) >>> # Data after >>> # | x | y |z | >>> # +---------+-----------------+---------+ >>> # | [0,1,2,3] | [0,1,2,1,2,3] | [1,2,2,1] >>> # +---------+-----------------+---------+
tinyms.text¶
This module is to support text processing for NLP tasks. It is a high performance NLP text processing module which is developed with ICU4C and cppjieba.
-
class
tinyms.text.BertDatasetTransform[source]¶ Apply preprocess operation on GeneratorDataset instance.
-
class
tinyms.text.Lookup(vocab, unknown_token=None, data_type=mindspore.int32)[source]¶ Look up a word into an id according to the input vocabulary table.
- Parameters
vocab (Vocab) – A vocabulary object.
unknown_token (str, optional) – Word used for lookup if the word being looked up is out-of-vocabulary (OOV). If unknown_token is OOV, a runtime error will be thrown (default=None).
data_type (mindspore.dtype, optional) – mindspore.dtype that lookup maps string to (default=mindspore.int32)
Examples
>>> # Load vocabulary from list >>> vocab = text.Vocab.from_list(['深', '圳', '欢', '迎', '您']) >>> # Use Lookup operator to map tokens to ids >>> lookup = text.Lookup(vocab) >>> text_file_dataset = text_file_dataset.map(operations=[lookup])
-
class
tinyms.text.JiebaTokenizer(hmm_path, mp_path, mode=<JiebaMode.MIX: 0>, with_offsets=False)[source]¶ Tokenize Chinese string into words based on dictionary.
Note
The integrity of the HMMSEgment algorithm and MPSegment algorithm files must be confirmed.
- Parameters
hmm_path (str) – Dictionary file is used by HMMSegment algorithm. The dictionary can be obtained on the official website of cppjieba.
mp_path (str) – Dictionary file is used by MPSegment algorithm. The dictionary can be obtained on the official website of cppjieba.
mode (JiebaMode, optional) –
Valid values can be any of [JiebaMode.MP, JiebaMode.HMM, JiebaMode.MIX](default=JiebaMode.MIX).
JiebaMode.MP, tokenize with MPSegment algorithm.
JiebaMode.HMM, tokenize with Hiddel Markov Model Segment algorithm.
JiebaMode.MIX, tokenize with a mix of MPSegment and HMMSegment algorithm.
with_offsets (bool, optional) – If or not output offsets of tokens (default=False).
Examples
>>> from mindspore.dataset.text import JiebaMode >>> # If with_offsets=False, default output one column {["text", dtype=str]} >>> jieba_hmm_file = "/path/to/jieba/hmm/file" >>> jieba_mp_file = "/path/to/jieba/mp/file" >>> tokenizer_op = text.JiebaTokenizer(jieba_hmm_file, jieba_mp_file, mode=JiebaMode.MP, with_offsets=False) >>> text_file_dataset = text_file_dataset.map(operations=tokenizer_op) >>> # If with_offsets=False, then output three columns {["token", dtype=str], ["offsets_start", dtype=uint32], >>> # ["offsets_limit", dtype=uint32]} >>> tokenizer_op = text.JiebaTokenizer(jieba_hmm_file, jieba_mp_file, mode=JiebaMode.MP, with_offsets=True) >>> text_file_dataset_1 = text_file_dataset_1.map(operations=tokenizer_op, input_columns=["text"], ... output_columns=["token", "offsets_start", "offsets_limit"], ... column_order=["token", "offsets_start", "offsets_limit"])
-
add_dict(user_dict)[source]¶ Add user defined word to JiebaTokenizer’s dictionary.
- Parameters
user_dict (Union[str, dict]) –
One of the two loading methods is file path(str) loading (according to the Jieba dictionary format) and the other is Python dictionary(dict) loading, Python Dict format: {word1:freq1, word2:freq2,…}. Jieba dictionary format : word(required), freq(optional), such as:
word1 freq1 word2 None word3 freq3
Examples
>>> from mindspore.dataset.text import JiebaMode >>> jieba_hmm_file = "/path/to/jieba/hmm/file" >>> jieba_mp_file = "/path/to/jieba/mp/file" >>> user_dict = {"男默女泪": 10} >>> jieba_op = text.JiebaTokenizer(jieba_hmm_file, jieba_mp_file, mode=JiebaMode.MP) >>> jieba_op.add_dict(user_dict) >>> text_file_dataset = text_file_dataset.map(operations=jieba_op, input_columns=["text"])
-
add_word(word, freq=None)[source]¶ Add user defined word to JiebaTokenizer’s dictionary.
- Parameters
word (str) – The word to be added to the JiebaTokenizer instance. The added word will not be written into the built-in dictionary on disk.
freq (int, optional) – The frequency of the word to be added. The higher the frequency, the better chance the word will be tokenized (default=None, use default frequency).
Examples
>>> from mindspore.dataset.text import JiebaMode >>> jieba_hmm_file = "/path/to/jieba/hmm/file" >>> jieba_mp_file = "/path/to/jieba/mp/file" >>> jieba_op = text.JiebaTokenizer(jieba_hmm_file, jieba_mp_file, mode=JiebaMode.MP) >>> sentence_piece_vocab_file = "/path/to/sentence/piece/vocab/file" >>> with open(sentence_piece_vocab_file, 'r') as f: ... for line in f: ... word = line.split(',')[0] ... jieba_op.add_word(word) >>> text_file_dataset = text_file_dataset.map(operations=jieba_op, input_columns=["text"])
-
class
tinyms.text.UnicodeCharTokenizer(with_offsets=False)[source]¶ Tokenize a scalar tensor of UTF-8 string to Unicode characters.
- Parameters
with_offsets (bool, optional) – If or not output offsets of tokens (default=False).
Examples
>>> # If with_offsets=False, default output one column {["text", dtype=str]} >>> tokenizer_op = text.UnicodeCharTokenizer(with_offsets=False) >>> text_file_dataset = text_file_dataset.map(operations=tokenizer_op) >>> # If with_offsets=True, then output three columns {["token", dtype=str], ["offsets_start", dtype=uint32], >>> # ["offsets_limit", dtype=uint32]} >>> tokenizer_op = text.UnicodeCharTokenizer(with_offsets=True) >>> text_file_dataset = text_file_dataset.map(operations=tokenizer_op, input_columns=["text"], ... output_columns=["token", "offsets_start", "offsets_limit"], ... column_order=["token", "offsets_start", "offsets_limit"])
-
class
tinyms.text.Ngram(n, left_pad=('', 0), right_pad=('', 0), separator=' ')[source]¶ TensorOp to generate n-gram from a 1-D string Tensor.
Refer to https://en.wikipedia.org/wiki/N-gram#Examples for an overview of what n-gram is and how it works.
- Parameters
n (list[int]) – n in n-gram, which is a list of positive integers. For example, if n=[4, 3], then the result would be a 4-gram followed by a 3-gram in the same tensor. If the number of words is not enough to make up for a n-gram, an empty string will be returned. For example, 3 grams on [“mindspore”, “best”] will result in an empty string produced.
left_pad (tuple, optional) – Padding performed on left side of the sequence shaped like (“pad_token”, pad_width). pad_width will be capped at n-1. For example, specifying left_pad=(“_”, 2) would pad left side of the sequence with “__” (default=None).
right_pad (tuple, optional) – Padding performed on right side of the sequence shaped like (“pad_token”, pad_width). pad_width will be capped at n-1. For example, specifying right_pad=(“_”, 2) would pad right side of the sequence with “__” (default=None).
separator (str, optional) – Symbol used to join strings together. For example. if 2-gram is [“mindspore”, “amazing”] with separator=”-“, the result would be [“mindspore-amazing”] (default=None, which will use whitespace as separator).
Examples
>>> ngram_op = text.Ngram(3, separator="-") >>> output = ngram_op(["WildRose Country", "Canada's Ocean Playground", "Land of Living Skies"]) >>> # output >>> # ["WildRose Country-Canada's Ocean Playground-Land of Living Skies"] >>> # same ngram_op called through map >>> text_file_dataset = text_file_dataset.map(operations=ngram_op)
-
class
tinyms.text.WordpieceTokenizer(vocab, suffix_indicator='##', max_bytes_per_token=100, unknown_token='[UNK]', with_offsets=False)[source]¶ Tokenize scalar token or 1-D tokens to 1-D subword tokens.
- Parameters
vocab (Vocab) – A vocabulary object.
suffix_indicator (str, optional) – Used to show that the subword is the last part of a word (default=’##’).
max_bytes_per_token (int, optional) – Tokens exceeding this length will not be further split (default=100).
unknown_token (str, optional) – When a token cannot be found: if ‘unknown_token’ is empty string, return the token directly, else return ‘unknown_token’ (default=’[UNK]’).
with_offsets (bool, optional) – If or not output offsets of tokens (default=False).
Examples
>>> vocab_list = ["book", "cholera", "era", "favor", "##ite", "my", "is", "love", "dur", "##ing", "the"] >>> vocab = text.Vocab.from_list(vocab_list) >>> # If with_offsets=False, default output one column {["text", dtype=str]} >>> tokenizer_op = text.WordpieceTokenizer(vocab=vocab, unknown_token='[UNK]', ... max_bytes_per_token=100, with_offsets=False) >>> text_file_dataset = text_file_dataset.map(operations=tokenizer_op) >>> # If with_offsets=True, then output three columns {["token", dtype=str], ["offsets_start", dtype=uint32], >>> # ["offsets_limit", dtype=uint32]} >>> tokenizer_op = text.WordpieceTokenizer(vocab=vocab, unknown_token='[UNK]', ... max_bytes_per_token=100, with_offsets=True) >>> text_file_dataset = text_file_dataset.map(operations=tokenizer_op, input_columns=["text"], ... output_columns=["token", "offsets_start", "offsets_limit"], ... column_order=["token", "offsets_start", "offsets_limit"])
-
class
tinyms.text.TruncateSequencePair(max_length)[source]¶ Truncate a pair of rank-1 tensors such that the total length is less than max_length.
This operation takes two input tensors and returns two output Tensors.
- Parameters
max_length (int) – Maximum length required.
Examples
>>> dataset = ds.NumpySlicesDataset(data={"col1": [[1, 2, 3]], "col2": [[4, 5]]}) >>> # Data before >>> # | col1 | col2 | >>> # +-----------+-----------| >>> # | [1, 2, 3] | [4, 5] | >>> # +-----------+-----------+ >>> truncate_sequence_pair_op = text.TruncateSequencePair(max_length=4) >>> dataset = dataset.map(operations=truncate_sequence_pair_op) >>> # Data after >>> # | col1 | col2 | >>> # +-----------+-----------+ >>> # | [1, 2] | [4, 5] | >>> # +-----------+-----------+
-
class
tinyms.text.ToNumber(data_type)[source]¶ Tensor operation to convert every element of a string tensor to a number.
Strings are casted according to the rules specified in the following links: https://en.cppreference.com/w/cpp/string/basic_string/stof, https://en.cppreference.com/w/cpp/string/basic_string/stoul, except that any strings which represent negative numbers cannot be cast to an unsigned integer type.
- Parameters
data_type (mindspore.dtype) – mindspore.dtype to be casted to. Must be a numeric type.
- Raises
RuntimeError – If strings are invalid to cast, or are out of range after being casted.
Examples
>>> import mindspore.common.dtype as mstype >>> data = [["1", "2", "3"]] >>> dataset = ds.NumpySlicesDataset(data) >>> to_number_op = text.ToNumber(mstype.int8) >>> dataset = dataset.map(operations=to_number_op)
-
class
tinyms.text.SlidingWindow(width, axis=0)[source]¶ TensorOp to construct a tensor from data (only 1-D for now), where each element in the dimension axis is a slice of data starting at the corresponding position, with a specified width.
- Parameters
Examples
>>> dataset = ds.NumpySlicesDataset(data=[[1, 2, 3, 4, 5]], column_names="col1") >>> # Data before >>> # | col1 | >>> # +--------------+ >>> # | [[1, 2, 3, 4, 5]] | >>> # +--------------+ >>> dataset = dataset.map(operations=text.SlidingWindow(3, 0)) >>> # Data after >>> # | col1 | >>> # +--------------+ >>> # | [[1, 2, 3], | >>> # | [2, 3, 4], | >>> # | [3, 4, 5]] | >>> # +--------------+
-
class
tinyms.text.SentencePieceTokenizer(mode, out_type)[source]¶ Tokenize scalar token or 1-D tokens to tokens by sentencepiece.
- Parameters
mode (Union[str, SentencePieceVocab]) – If the input parameter is a file, then it is of type string. If the input parameter is a SentencePieceVocab object, then it is of type SentencePieceVocab.
Examples
>>> from mindspore.dataset.text import SentencePieceModel, SPieceTokenizerOutType >>> sentence_piece_vocab_file = "/path/to/sentence/piece/vocab/file" >>> vocab = text.SentencePieceVocab.from_file([sentence_piece_vocab_file], 5000, 0.9995, ... SentencePieceModel.UNIGRAM, {}) >>> tokenizer = text.SentencePieceTokenizer(vocab, out_type=SPieceTokenizerOutType.STRING) >>> text_file_dataset = text_file_dataset.map(operations=tokenizer)
-
class
tinyms.text.PythonTokenizer(tokenizer)[source]¶ Callable class to be used for user-defined string tokenizer.
- Parameters
tokenizer (Callable) – Python function that takes a str and returns a list of str as tokens.
Examples
>>> def my_tokenizer(line): ... return line.split() >>> text_file_dataset = text_file_dataset.map(operations=text.PythonTokenizer(my_tokenizer))
-
tinyms.text.to_str(array, encoding='utf8')[source]¶ Convert NumPy array of bytes to array of str by decoding each element based on charset encoding.
- Parameters
array (numpy.ndarray) – Array of type bytes representing strings.
encoding (str) – Indicating the charset for decoding.
- Returns
numpy.ndarray, NumPy array of str.
-
tinyms.text.to_bytes(array, encoding='utf8')[source]¶ Convert NumPy array of str to array of bytes by encoding each element based on charset encoding.
- Parameters
array (numpy.ndarray) – Array of type str representing strings.
encoding (str) – Indicating the charset for encoding.
- Returns
numpy.ndarray, NumPy array of bytes.
-
class
tinyms.text.Vocab[source]¶ Vocab object that is used to lookup a word.
It contains a map that maps each word(str) to an id (int).
-
classmethod
from_dataset(dataset, columns=None, freq_range=None, top_k=None, special_tokens=None, special_first=True)[source]¶ Build a vocab from a dataset.
This would collect all unique words in a dataset and return a vocab within the frequency range specified by user in freq_range. User would be warned if no words fall into the frequency. Words in vocab are ordered from highest frequency to lowest frequency. Words with the same frequency would be ordered lexicographically.
- Parameters
dataset (Dataset) – dataset to build vocab from.
columns (list[str], optional) – column names to get words from. It can be a list of column names. (default=None, where all columns will be used. If any column isn’t string type, will return error).
freq_range (tuple, optional) – A tuple of integers (min_frequency, max_frequency). Words within the frequency range would be kept. 0 <= min_frequency <= max_frequency <= total_words. min_frequency=0 is the same as min_frequency=1. max_frequency > total_words is the same as max_frequency = total_words. min_frequency/max_frequency can be None, which corresponds to 0/total_words separately (default=None, all words are included).
top_k (int, optional) – top_k > 0. Number of words to be built into vocab. top_k most frequent words are taken. top_k is taken after freq_range. If not enough top_k, all words will be taken (default=None, all words are included).
special_tokens (list, optional) – a list of strings, each one is a special token. for example special_tokens=[“<pad>”,”<unk>”] (default=None, no special tokens will be added).
special_first (bool, optional) – whether special_tokens will be prepended/appended to vocab. If special_tokens is specified and special_first is set to True, special_tokens will be prepended (default=True).
- Returns
Vocab, vocab built from the dataset.
-
classmethod
from_dict(word_dict)[source]¶ Build a vocab object from a dict.
- Parameters
word_dict (dict) – dict contains word and id pairs, where word should be str and id be int. id is recommended to start from 0 and be continuous. ValueError will be raised if id is negative.
- Returns
Vocab, vocab built from the dict.
-
classmethod
from_file(file_path, delimiter='', vocab_size=None, special_tokens=None, special_first=True)[source]¶ Build a vocab object from a list of word.
- Parameters
file_path (str) – path to the file which contains the vocab list.
delimiter (str, optional) – a delimiter to break up each line in file, the first element is taken to be the word (default=””).
vocab_size (int, optional) – number of words to read from file_path (default=None, all words are taken).
special_tokens (list, optional) – a list of strings, each one is a special token. for example special_tokens=[“<pad>”,”<unk>”] (default=None, no special tokens will be added).
special_first (bool, optional) – whether special_tokens will be prepended/appended to vocab, If special_tokens is specified and special_first is set to True, special_tokens will be prepended (default=True).
- Returns
Vocab, vocab built from the file.
-
classmethod
from_list(word_list, special_tokens=None, special_first=True)[source]¶ Build a vocab object from a list of word.
- Parameters
word_list (list) – a list of string where each element is a word of type string.
special_tokens (list, optional) – a list of strings, each one is a special token. for example special_tokens=[“<pad>”,”<unk>”] (default=None, no special tokens will be added).
special_first (bool, optional) – whether special_tokens will be prepended/appended to vocab, If special_tokens is specified and special_first is set to True, special_tokens will be prepended (default=True).
- Returns
Vocab, vocab built from the list.
-
classmethod
-
class
tinyms.text.SentencePieceVocab[source]¶ SentencePiece obiect that is used to segmentate words
-
classmethod
from_dataset(dataset, col_names, vocab_size, character_coverage, model_type, params)[source]¶ Build a sentencepiece from a dataset
- Parameters
dataset (Dataset) – Dataset to build sentencepiece.
col_names (list) – The list of the col name.
vocab_size (int) – Vocabulary size.
character_coverage (float) – Amount of characters covered by the model, good defaults are: 0.9995 for languages. with rich character set like Japanese or Chinese and 1.0 for other languages with small character set.
model_type (SentencePieceModel) – Choose from UNIGRAM (default), BPE, CHAR, or WORD. The input sentence must be pretokenized when using word type.
params (dict) – A dictionary with no incoming parameters.
- Returns
SentencePieceVocab, vocab built from the dataset.
-
classmethod
from_file(file_path, vocab_size, character_coverage, model_type, params)[source]¶ Build a SentencePiece object from a list of word.
- Parameters
file_path (list) – Path to the file which contains the sentencepiece list.
vocab_size (int) – Vocabulary size, the type of uint32_t.
character_coverage (float) – Amount of characters covered by the model, good defaults are: 0.9995 for languages. with rich character set like Japanse or Chinese and 1.0 for other languages with small character set.
model_type (SentencePieceModel) – Choose from unigram (default), bpe, char, or word. The input sentence must be pretokenized when using word type.
params (dict) –
A dictionary with no incoming parameters(The parameters are derived from SentencePiece library).
input_sentence_size 0 max_sentencepiece_length 16
- Returns
SentencePieceVocab, vocab built from the file.
-
classmethod
save_model(vocab, path, filename)[source]¶ Save model to filepath
- Parameters
vocab (SentencePieceVocab) – A sentencepiece object.
path (str) – Path to store model.
filename (str) – The name of the file.
-
classmethod
-
class
tinyms.text.SentencePieceModel[source]¶ An enumeration for SentencePieceModel, effective enumeration types are UNIGRAM, BPE, CHAR, WORD.
-
class
tinyms.text.SPieceTokenizerOutType[source]¶ An enumeration for SPieceTokenizerOutType, effective enumeration types are STRING, INT.
-
class
tinyms.text.SPieceTokenizerLoadType[source]¶ An enumeration for SPieceTokenizerLoadType, effective enumeration types are FILE, MODEL.
-
class
tinyms.text.Compose(transforms)[source]¶ Compose a list of transforms into a single transform.
- Parameters
transforms (list) – List of transformations to be applied.
Examples
>>> compose = c_transforms.Compose([c_vision.Decode(), c_vision.RandomCrop(512)]) >>> image_folder_dataset = image_folder_dataset.map(operations=compose)
-
class
tinyms.text.Concatenate(axis=0, prepend=None, append=None)[source]¶ Tensor operation that concatenates all columns into a single tensor.
- Parameters
axis (int, optional) – Concatenate the tensors along given axis (Default=0).
prepend (numpy.array, optional) – NumPy array to be prepended to the already concatenated tensors (Default=None).
append (numpy.array, optional) – NumPy array to be appended to the already concatenated tensors (Default=None).
Examples
>>> import numpy as np >>> # concatenate string >>> prepend_tensor = np.array(["dw", "df"], dtype='S') >>> append_tensor = np.array(["dwsdf", "df"], dtype='S') >>> concatenate_op = c_transforms.Concatenate(0, prepend_tensor, append_tensor) >>> data = [["This","is","a","string"]] >>> dataset = ds.NumpySlicesDataset(data) >>> dataset = dataset.map(operations=concatenate_op)
-
class
tinyms.text.Duplicate[source]¶ Duplicate the input tensor to output, only support transform one column each time.
Examples
>>> # Data before >>> # | x | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------+ >>> data = [[1,2,3]] >>> numpy_slices_dataset = ds.NumpySlicesDataset(data, ["x"]) >>> numpy_slices_dataset = numpy_slices_dataset.map(operations=c_transforms.Duplicate(), ... input_columns=["x"], ... output_columns=["x", "y"], ... column_order=["x", "y"]) >>> # Data after >>> # | x | y | >>> # +---------+---------+ >>> # | [1,2,3] | [1,2,3] | >>> # +---------+---------+
-
class
tinyms.text.Fill(fill_value)[source]¶ Tensor operation to fill all elements in the tensor with the specified value. The output tensor will have the same shape and type as the input tensor.
- Parameters
fill_value (Union[str, bytes, int, float, bool])) – scalar value to fill the tensor with.
Examples
>>> import numpy as np >>> # generate a 1D integer numpy array from 0 to 4 >>> def generator_1d(): ... for i in range(5): ... yield (np.array([i]),) >>> generator_dataset = ds.GeneratorDataset(generator_1d, column_names="col1") >>> # [[0], [1], [2], [3], [4]] >>> fill_op = c_transforms.Fill(3) >>> generator_dataset = generator_dataset.map(operations=fill_op) >>> # [[3], [3], [3], [3], [3]]
-
class
tinyms.text.Mask(operator, constant, dtype=mindspore.bool)[source]¶ Mask content of the input tensor with the given predicate. Any element of the tensor that matches the predicate will be evaluated to True, otherwise False.
- Parameters
Examples
>>> from mindspore.dataset.transforms.c_transforms import Relational >>> # Data before >>> # | col | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------+ >>> data = [[1, 2, 3]] >>> numpy_slices_dataset = ds.NumpySlicesDataset(data, ["col"]) >>> numpy_slices_dataset = numpy_slices_dataset.map(operations=c_transforms.Mask(Relational.EQ, 2)) >>> # Data after >>> # | col | >>> # +--------------------+ >>> # | [False,True,False] | >>> # +--------------------+
-
class
tinyms.text.OneHot(num_classes)[source]¶ Tensor operation to apply one hot encoding.
- Parameters
num_classes (int) – Number of classes of objects in dataset. It should be larger than the largest label number in the dataset.
- Raises
RuntimeError – feature size is bigger than num_classes.
Examples
>>> # Assume that dataset has 10 classes, thus the label ranges from 0 to 9 >>> onehot_op = c_transforms.OneHot(num_classes=10) >>> mnist_dataset = mnist_dataset.map(operations=onehot_op, input_columns=["label"])
-
class
tinyms.text.PadEnd(pad_shape, pad_value=None)[source]¶ Pad input tensor according to pad_shape, need to have same rank.
- Parameters
pad_shape (list(int)) – List of integers representing the shape needed. Dimensions that set to None will not be padded (i.e., original dim will be used). Shorter dimensions will truncate the values.
pad_value (Union[str, bytes, int, float, bool]), optional) – Value used to pad. Default to 0 or empty string in case of tensors of strings.
Examples
>>> # Data before >>> # | col | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------| >>> data = [[1, 2, 3]] >>> numpy_slices_dataset = ds.NumpySlicesDataset(data, ["col"]) >>> numpy_slices_dataset = numpy_slices_dataset.map(operations=c_transforms.PadEnd(pad_shape=[4], ... pad_value=10)) >>> # Data after >>> # | col | >>> # +------------+ >>> # | [1,2,3,10] | >>> # +------------|
-
class
tinyms.text.RandomApply(transforms, prob=0.5)[source]¶ Randomly perform a series of transforms with a given probability.
- Parameters
Examples
>>> rand_apply = c_transforms.RandomApply([c_vision.RandomCrop(512)]) >>> image_folder_dataset = image_folder_dataset.map(operations=rand_apply)
-
class
tinyms.text.RandomChoice(transforms)[source]¶ Randomly select one transform from a list of transforms to perform operation.
- Parameters
transforms (list) – List of transformations to be chosen from to apply.
Examples
>>> rand_choice = c_transforms.RandomChoice([c_vision.CenterCrop(50), c_vision.RandomCrop(512)]) >>> image_folder_dataset = image_folder_dataset.map(operations=rand_choice)
-
class
tinyms.text.Slice(*slices)[source]¶ Slice operation to extract a tensor out using the given n slices.
The functionality of Slice is similar to NumPy’s indexing feature. (Currently only rank-1 tensors are supported).
- Parameters
slices (Union[int, list[int], slice, None, Ellipsis]) –
Maximum n number of arguments to slice a tensor of rank n . One object in slices can be one of:
int: Slice this index only along the first dimension. Negative index is supported.list(int): Slice these indices along the first dimension. Negative indices are supported.slice: Slice the generated indices from the slice object along the first dimension. Similar to start:stop:step.None: Slice the whole dimension. Similar to[:]in Python indexing.Ellipsis: Slice the whole dimension, same result with None.
Examples
>>> # Data before >>> # | col | >>> # +---------+ >>> # | [1,2,3] | >>> # +---------| >>> data = [[1, 2, 3]] >>> numpy_slices_dataset = ds.NumpySlicesDataset(data, ["col"]) >>> # slice indices 1 and 2 only >>> numpy_slices_dataset = numpy_slices_dataset.map(operations=c_transforms.Slice(slice(1,3))) >>> # Data after >>> # | col | >>> # +---------+ >>> # | [2,3] | >>> # +---------|
-
class
tinyms.text.TypeCast(data_type)[source]¶ Tensor operation to cast to a given MindSpore data type.
- Parameters
data_type (mindspore.dtype) – mindspore.dtype to be cast to.
Examples
>>> import numpy as np >>> import mindspore.common.dtype as mstype >>> >>> # Generate 1d int numpy array from 0 - 63 >>> def generator_1d(): ... for i in range(64): ... yield (np.array([i]),) >>> >>> dataset = ds.GeneratorDataset(generator_1d, column_names='col') >>> type_cast_op = c_transforms.TypeCast(mstype.int32) >>> dataset = dataset.map(operations=type_cast_op)
-
class
tinyms.text.Unique[source]¶ Perform the unique operation on the input tensor, only support transform one column each time.
Return 3 tensor: unique output tensor, index tensor, count tensor.
Unique output tensor contains all the unique elements of the input tensor in the same order that they occur in the input tensor.
Index tensor that contains the index of each element of the input tensor in the unique output tensor.
Count tensor that contains the count of each element of the output tensor in the input tensor.
Note
Call batch op before calling this function.
Examples
>>> # Data before >>> # | x | >>> # +--------------------+ >>> # | [[0,1,2], [1,2,3]] | >>> # +--------------------+ >>> data = [[[0,1,2], [1,2,3]]] >>> dataset = ds.NumpySlicesDataset(data, ["x"]) >>> dataset = dataset.map(operations=c_transforms.Unique(), ... input_columns=["x"], ... output_columns=["x", "y", "z"], ... column_order=["x", "y", "z"]) >>> # Data after >>> # | x | y |z | >>> # +---------+-----------------+---------+ >>> # | [0,1,2,3] | [0,1,2,1,2,3] | [1,2,2,1] >>> # +---------+-----------------+---------+
tinyms.primitives¶
Primitives module. Operators can be used in the construct function of Layer.
Examples
>>> import tinyms as ts
>>> from tinyms.primitives import tensor_add
>>>
>>> x = ts.ones([2, 3])
>>> y = ts.ones([2, 3])
>>> print(tensor_add(x, y))
[[2. 2. 2.]
[2. 2. 2.]]
-
tinyms.primitives.core(fn=None, **flags)[source]¶ A decorator that adds a flag to the function.
By default, the function is marked as True, enabling to use this decorator to set flag to a graph.
- Parameters
fn (Function) – Function to add flag. Default: None.
flags (dict) – The following flags can be set core, which indicates that this is a core function or other flag. Default: None.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> net = core(net, predit=True) >>> print(hasattr(net, '_mindspore_flags')) True
-
tinyms.primitives.add_flags(fn=None, **flags)[source]¶ A decorator that adds a flag to the function.
Note
Only supports bool value.
- Parameters
fn (Function) – Function or cell to add flag. Default: None.
flags (dict) – Flags use kwargs. Default: None.
- Returns
Function, the function with added flags.
Examples
>>> net = Net(); >>> net = add_flags(net, predit=True) >>> print(hasattr(net, '_mindspore_flags')) True
-
class
tinyms.primitives.MultitypeFuncGraph(name, read_value=False)[source]¶ Generates overloaded functions.
MultitypeFuncGraph is a class used to generate overloaded functions, considering different types as inputs. Initialize an MultitypeFuncGraph object with name, and use register with input types as the decorator for the function to be registered. And the object can be called with different types of inputs, and work with HyperMap and Map.
- Parameters
- Raises
ValueError – If failed to find find a matching function for the given arguments.
- Supported Platforms:
AscendGPUCPU
Examples
>>> # `add` is a metagraph object which will add two objects according to >>> # input type using ".register" decorator. >>> from mindspore import Tensor >>> from mindspore.ops import Primitive, operations as P >>> from mindspore import dtype as mstype >>> >>> tensor_add = P.Add() >>> add = MultitypeFuncGraph('add') >>> @add.register("Number", "Number") ... def add_scala(x, y): ... return x + y >>> @add.register("Tensor", "Tensor") ... def add_tensor(x, y): ... return tensor_add(x, y) >>> output = add(1, 2) >>> print(output) 3 >>> output = add(Tensor([0.1, 0.6, 1.2], dtype=mstype.float32), Tensor([0.1, 0.6, 1.2], dtype=mstype.float32)) >>> print(output) [0.2 1.2 2.4]
-
class
tinyms.primitives.GradOperation(get_all=False, get_by_list=False, sens_param=False)[source]¶ A higher-order function which is used to generate the gradient function for the input function.
The gradient function generated by GradOperation higher-order function can be customized by construction arguments.
Given an input function net = Net() that takes x and y as inputs, and has a parameter z, see Net in Examples.
To generate a gradient function that returns gradients with respect to the first input (see GradNetWrtX in Examples).
Construct a GradOperation higher-order function with default arguments: grad_op = GradOperation().
Call it with input function as argument to get the gradient function: gradient_function = grad_op(net).
Call the gradient function with input function’s inputs to get the gradients with respect to the first input: grad_op(net)(x, y).
To generate a gradient function that returns gradients with respect to all inputs (see GradNetWrtXY in Examples).
Construct a GradOperation higher-order function with get_all=True which indicates getting gradients with respect to all inputs, they are x and y in example function Net(): grad_op = GradOperation(get_all=True).
Call it with input function as argument to get the gradient function: gradient_function = grad_op(net).
Call the gradient function with input function’s inputs to get the gradients with respect to all inputs: gradient_function(x, y).
To generate a gradient function that returns gradients with respect to given parameters (see GradNetWithWrtParams in Examples).
Construct a GradOperation higher-order function with get_by_list=True: grad_op = GradOperation(get_by_list=True).
Construct a ParameterTuple that will be passed to the input function when constructing GradOperation higher-order function, it will be used as a parameter filter that determine which gradient to return: params = ParameterTuple(net.trainable_params()).
Call it with input function and params as arguments to get the gradient function: gradient_function = grad_op(net, params).
Call the gradient function with input function’s inputs to get the gradients with respect to given parameters: gradient_function(x, y).
To generate a gradient function that returns gradients with respect to all inputs and given parameters in the format of ((dx, dy), (dz))(see GradNetWrtInputsAndParams in Examples).
Construct a GradOperation higher-order function with get_all=True and get_by_list=True: grad_op = GradOperation(get_all=True, get_by_list=True).
Construct a ParameterTuple that will be passed along input function when constructing GradOperation higher-order function: params = ParameterTuple(net.trainable_params()).
Call it with input function and params as arguments to get the gradient function: gradient_function = grad_op(net, params).
Call the gradient function with input function’s inputs to get the gradients with respect to all inputs and given parameters: gradient_function(x, y).
We can configure the sensitivity(gradient with respect to output) by setting sens_param as True and passing an extra sensitivity input to the gradient function, the sensitivity input should has the same shape and type with input function’s output(see GradNetWrtXYWithSensParam in Examples).
Construct a GradOperation higher-order function with get_all=True and sens_param=True: grad_op = GradOperation(get_all=True, sens_param=True).
Define grad_wrt_output as sens_param which works as the gradient with respect to output: grad_wrt_output = Tensor(np.ones([2, 2]).astype(np.float32)).
Call it with input function as argument to get the gradient function: gradient_function = grad_op(net).
Call the gradient function with input function’s inputs and sens_param to get the gradients with respect to all inputs: gradient_function(x, y, grad_wrt_output).
- Parameters
get_all (bool) – If True, get all the gradients with respect to inputs. Default: False.
get_by_list (bool) – If True, get all the gradients with respect to Parameter variables. If get_all and get_by_list are both False, get the gradient with respect to first input. If get_all and get_by_list are both True, get the gradients with respect to inputs and Parameter variables at the same time in the form of ((gradients with respect to inputs), (gradients with respect to parameters)). Default: False.
sens_param (bool) – Whether to append sensitivity (gradient with respect to output) as input. If sens_param is False, a ‘ones_like(outputs)’ sensitivity will be attached automatically. Default: False. If the sensor_param is True, a sensitivity (gradient with respect to output) needs to be transferred through the location parameter or key-value pair parameter. If the value is transferred through the key-value pair parameter, the key must be sens.
- Returns
The higher-order function which takes a function as argument and returns gradient function for it.
- Raises
TypeError – If get_all, get_by_list or sens_param is not a bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore.common import ParameterTuple >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.matmul = P.MatMul() ... self.z = Parameter(Tensor(np.array([1.0], np.float32)), name='z') ... def construct(self, x, y): ... x = x * self.z ... out = self.matmul(x, y) ... return out ... >>> class GradNetWrtX(nn.Cell): ... def __init__(self, net): ... super(GradNetWrtX, self).__init__() ... self.net = net ... self.grad_op = GradOperation() ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net) ... return gradient_function(x, y) ... >>> x = Tensor([[0.5, 0.6, 0.4], [1.2, 1.3, 1.1]], dtype=mstype.float32) >>> y = Tensor([[0.01, 0.3, 1.1], [0.1, 0.2, 1.3], [2.1, 1.2, 3.3]], dtype=mstype.float32) >>> output = GradNetWrtX(Net())(x, y) >>> print(output) [[1.4100001 1.5999999 6.6 ] [1.4100001 1.5999999 6.6 ]] >>> >>> class GradNetWrtXY(nn.Cell): ... def __init__(self, net): ... super(GradNetWrtXY, self).__init__() ... self.net = net ... self.grad_op = GradOperation(get_all=True) ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net) ... return gradient_function(x, y) >>> >>> x = Tensor([[0.8, 0.6, 0.2], [1.8, 1.3, 1.1]], dtype=mstype.float32) >>> y = Tensor([[0.11, 3.3, 1.1], [1.1, 0.2, 1.4], [1.1, 2.2, 0.3]], dtype=mstype.float32) >>> output = GradNetWrtXY(Net())(x, y) >>> print(output) (Tensor(shape=[2, 3], dtype=Float32, value= [[ 4.50999975e+00, 2.70000005e+00, 3.60000014e+00], [ 4.50999975e+00, 2.70000005e+00, 3.60000014e+00]]), Tensor(shape=[3, 3], dtype=Float32, value= [[ 2.59999990e+00, 2.59999990e+00, 2.59999990e+00], [ 1.89999998e+00, 1.89999998e+00, 1.89999998e+00], [ 1.30000007e+00, 1.30000007e+00, 1.30000007e+00]])) >>> >>> class GradNetWrtXYWithSensParam(nn.Cell): ... def __init__(self, net): ... super(GradNetWrtXYWithSensParam, self).__init__() ... self.net = net ... self.grad_op = GradOperation(get_all=True, sens_param=True) ... self.grad_wrt_output = Tensor([[0.1, 0.6, 0.2], [0.8, 1.3, 1.1]], dtype=mstype.float32) ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net) ... return gradient_function(x, y, self.grad_wrt_output) >>> >>> x = Tensor([[0.8, 0.6, 0.2], [1.8, 1.3, 1.1]], dtype=mstype.float32) >>> y = Tensor([[0.11, 3.3, 1.1], [1.1, 0.2, 1.4], [1.1, 2.2, 0.3]], dtype=mstype.float32) >>> output = GradNetWrtXYWithSensParam(Net())(x, y) >>> print(output) (Tensor(shape=[2, 3], dtype=Float32, value= [[ 2.21099997e+00, 5.09999990e-01, 1.49000001e+00], [ 5.58799982e+00, 2.68000007e+00, 4.07000017e+00]]), Tensor(shape=[3, 3], dtype=Float32, value= [[ 1.51999998e+00, 2.81999993e+00, 2.14000010e+00], [ 1.09999990e+00, 2.04999971e+00, 1.54999995e+00], [ 9.00000036e-01, 1.54999995e+00, 1.25000000e+00]])) >>> >>> class GradNetWithWrtParams(nn.Cell): ... def __init__(self, net): ... super(GradNetWithWrtParams, self).__init__() ... self.net = net ... self.params = ParameterTuple(net.trainable_params()) ... self.grad_op = GradOperation(get_by_list=True) ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net, self.params) ... return gradient_function(x, y) >>> >>> x = Tensor([[0.8, 0.6, 0.2], [1.8, 1.3, 1.1]], dtype=mstype.float32) >>> y = Tensor([[0.11, 3.3, 1.1], [1.1, 0.2, 1.4], [1.1, 2.2, 0.3]], dtype=mstype.float32) >>> output = GradNetWithWrtParams(Net())(x, y) >>> print(output) (Tensor(shape=[1], dtype=Float32, value= [ 2.15359993e+01]),) >>> >>> class GradNetWrtInputsAndParams(nn.Cell): ... def __init__(self, net): ... super(GradNetWrtInputsAndParams, self).__init__() ... self.net = net ... self.params = ParameterTuple(net.trainable_params()) ... self.grad_op = GradOperation(get_all=True, get_by_list=True) ... def construct(self, x, y): ... gradient_function = self.grad_op(self.net, self.params) ... return gradient_function(x, y) >>> >>> x = Tensor([[0.1, 0.6, 1.2], [0.5, 1.3, 0.1]], dtype=mstype.float32) >>> y = Tensor([[0.12, 2.3, 1.1], [1.3, 0.2, 2.4], [0.1, 2.2, 0.3]], dtype=mstype.float32) >>> output = GradNetWrtInputsAndParams(Net())(x, y) >>> print(output) ((Tensor(shape=[2, 3], dtype=Float32, value= [[ 3.51999998e+00, 3.90000010e+00, 2.59999990e+00], [ 3.51999998e+00, 3.90000010e+00, 2.59999990e+00]]), Tensor(shape=[3, 3], dtype=Float32, value= [[ 6.00000024e-01, 6.00000024e-01, 6.00000024e-01], [ 1.89999998e+00, 1.89999998e+00, 1.89999998e+00], [ 1.30000007e+00, 1.30000007e+00, 1.30000007e+00]])), (Tensor(shape=[1], dtype=Float32, value= [ 1.29020004e+01]),))
-
class
tinyms.primitives.HyperMap(ops=None)[source]¶ Hypermap will apply the set operation to input sequences.
Apply the operations to every elements of the sequence or nested sequence. Different from Map, the HyperMap supports to apply on nested structure.
- Parameters
ops (Union[MultitypeFuncGraph, None]) – ops is the operation to apply. If ops is None, the operations should be put in the first input of the instance.
- Inputs:
args (Tuple[sequence]) - If ops is None, all the inputs should be sequences with the same length. And each row of the sequences will be the inputs of the operation.
If ops is not None, the first input is the operation, and the others are inputs.
- Outputs:
Sequence or nested sequence, the sequence of output after applying the function. e.g. operation(args[0][i], args[1][i]).
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore import dtype as mstype >>> nest_tensor_list = ((Tensor(1, mstype.float32), Tensor(2, mstype.float32)), ... (Tensor(3, mstype.float32), Tensor(4, mstype.float32))) >>> # square all the tensor in the nested list >>> >>> square = MultitypeFuncGraph('square') >>> @square.register("Tensor") ... def square_tensor(x): ... return F.square(x) >>> >>> common_map = HyperMap() >>> output = common_map(square, nest_tensor_list) >>> print(output) ((Tensor(shape=[], dtype=Float32, value= 1), Tensor(shape=[], dtype=Float32, value= 4)), (Tensor(shape=[], dtype=Float32, value= 9), Tensor(shape=[], dtype=Float32, value= 16))) >>> square_map = HyperMap(square) >>> output = square_map(nest_tensor_list) >>> print(output) ((Tensor(shape=[], dtype=Float32, value= 1), Tensor(shape=[], dtype=Float32, value= 4)), (Tensor(shape=[], dtype=Float32, value= 9), Tensor(shape=[], dtype=Float32, value= 16)))
-
tinyms.primitives.normal(shape, mean, stddev, seed=None)[source]¶ Generates random numbers according to the Normal (or Gaussian) random number distribution.
- Parameters
shape (tuple) – The shape of random tensor to be generated.
mean (Tensor) – The mean μ distribution parameter, which specifies the location of the peak, with data type in [int8, int16, int32, int64, float16, float32].
stddev (Tensor) – The deviation σ distribution parameter. It should be greater than 0, with data type in [int8, int16, int32, int64, float16, float32].
seed (int) – Seed is used as entropy source for the Random number engines to generate pseudo-random numbers. must be non-negative. Default: None, which will be treated as 0.
- Returns
Tensor. The shape should be equal to the broadcasted shape between the input shape and shapes of mean and stddev. The dtype is float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> shape = (3, 1, 2) >>> mean = Tensor(np.array([[3, 4], [5, 6]]), mstype.float32) >>> stddev = Tensor(1.0, mstype.float32) >>> output = C.normal(shape, mean, stddev, seed=5) >>> result = output.shape >>> print(result) (3, 2, 2)
-
tinyms.primitives.laplace(shape, mean, lambda_param, seed=None)[source]¶ Generates random numbers according to the Laplace random number distribution. It is defined as:
\[\text{f}(x;μ,λ) = \frac{1}{2λ}\exp(-\frac{|x-μ|}{λ}),\]- Parameters
shape (tuple) – The shape of random tensor to be generated.
mean (Tensor) – The mean μ distribution parameter, which specifies the location of the peak. With float32 data type.
lambda_param (Tensor) – The parameter used for controlling the variance of this random distribution. The variance of Laplace distribution is equal to twice the square of lambda_param. With float32 data type.
seed (int) – Seed is used as entropy source for Random number engines generating pseudo-random numbers. Default: None, which will be treated as 0.
- Returns
Tensor. The shape should be the broadcasted shape of Input “shape” and shapes of mean and lambda_param. The dtype is float32.
- Supported Platforms:
Ascend
Examples
>>> from mindspore import Tensor >>> from mindspore.ops import composite as C >>> import mindspore.common.dtype as mstype >>> shape = (2, 3) >>> mean = Tensor(1.0, mstype.float32) >>> lambda_param = Tensor(1.0, mstype.float32) >>> output = C.laplace(shape, mean, lambda_param, seed=5) >>> print(output.shape) (2, 3)
-
tinyms.primitives.uniform(shape, minval, maxval, seed=None, dtype=mindspore.float32)[source]¶ Generates random numbers according to the Uniform random number distribution.
Note
The number in tensor minval should be strictly less than maxval at any position after broadcasting.
- Parameters
shape (tuple) – The shape of random tensor to be generated.
minval (Tensor) – The distribution parameter a. It defines the minimum possible generated value, with int32 or float32 data type. If dtype is int32, only one number is allowed.
maxval (Tensor) – The distribution parameter b. It defines the maximum possible generated value, with int32 or float32 data type. If dtype is int32, only one number is allowed.
seed (int) – Seed is used as entropy source for the random number engines to generate pseudo-random numbers, must be non-negative. Default: None, which will be treated as 0.
dtype (mindspore.dtype) – type of the Uniform distribution. If it is int32, it generates numbers from discrete uniform distribution; if it is float32, it generates numbers from continuous uniform distribution. It only supports these two data types. Default: mstype.float32.
- Returns
Tensor. The shape should be equal to the broadcasted shape between the input shape and shapes of minval and maxval. The dtype is designated as the input dtype.
- Supported Platforms:
AscendGPU
Examples
>>> # For discrete uniform distribution, only one number is allowed for both minval and maxval: >>> shape = (4, 2) >>> minval = Tensor(1, mstype.int32) >>> maxval = Tensor(2, mstype.int32) >>> output = C.uniform(shape, minval, maxval, seed=5, dtype=mstype.int32) >>> >>> # For continuous uniform distribution, minval and maxval can be multi-dimentional: >>> shape = (3, 1, 2) >>> minval = Tensor(np.array([[3, 4], [5, 6]]), mstype.float32) >>> maxval = Tensor([8.0, 10.0], mstype.float32) >>> output = C.uniform(shape, minval, maxval, seed=5) >>> result = output.shape >>> print(result) (3, 2, 2)
-
tinyms.primitives.gamma(shape, alpha, beta, seed=None)[source]¶ Generates random numbers according to the Gamma random number distribution.
- Parameters
shape (tuple) – The shape of random tensor to be generated.
alpha (Tensor) – The alpha α distribution parameter. It should be greater than 0 with float32 data type.
beta (Tensor) – The beta β distribution parameter. It should be greater than 0 with float32 data type.
seed (int) – Seed is used as entropy source for the random number engines to generate pseudo-random numbers, must be non-negative. Default: None, which will be treated as 0.
- Returns
Tensor. The shape should be equal to the broadcasted shape between the input “shape” and shapes of alpha and beta. The dtype is float32.
- Raises
- Supported Platforms:
Ascend
Examples
>>> shape = (3, 1, 2) >>> alpha = Tensor(np.array([[3, 4], [5, 6]]), mstype.float32) >>> beta = Tensor(np.array([1.0]), mstype.float32) >>> output = C.gamma(shape, alpha, beta, seed=5) >>> result = output.shape >>> print(result) (3, 2, 2)
-
tinyms.primitives.poisson(shape, mean, seed=None)[source]¶ Generates random numbers according to the Poisson random number distribution.
\[\text{P}(i|μ) = \frac{\exp(-μ)μ^{i}}{i!}\]- Parameters
shape (tuple) – The shape of random tensor to be generated.
mean (Tensor) – The mean μ distribution parameter. It should be greater than 0 with float32 data type.
seed (int) – Seed is used as entropy source for the random number engines to generate pseudo-random numbers and must be non-negative. Default: None, which will be treated as 0.
- Returns
Tensor. The shape should be equal to the broadcasted shape between the input “shape” and shapes of mean. The dtype is float32.
- Raises
- Supported Platforms:
Ascend
Examples
>>> shape = (4, 1) >>> mean = Tensor(np.array([5.0, 10.0]), mstype.float32) >>> output = C.poisson(shape, mean, seed=5) >>> result = output.shape >>> print(result) (4, 2)
-
tinyms.primitives.multinomial(inputs, num_sample, replacement=True, seed=None)[source]¶ Returns a tensor sampled from the multinomial probability distribution located in the corresponding row of the input tensor.
Note
The rows of input do not need to sum to one (in which case we use the values as weights), but must be non-negative, finite and have a non-zero sum.
- Parameters
inputs (Tensor) – The input tensor containing probabilities, must be 1 or 2 dimensions, with float32 data type.
num_sample (int) – Number of samples to draw.
replacement (bool, optional) – Whether to draw with replacement or not, default True.
seed (int, optional) – Seed is used as entropy source for the random number engines to generate pseudo-random numbers, must be non-negative. Default: 0.
- Outputs:
Tensor, has the same rows with input. The number of sampled indices of each row is num_samples. The dtype is float32.
- Raises
- Supported Platforms:
GPU
Examples
>>> input = Tensor([0, 9, 4, 0], mstype.float32) >>> output = C.multinomial(input, 2, True) >>> print(output) [1 1]
-
tinyms.primitives.clip_by_value(x, clip_value_min, clip_value_max)[source]¶ Clips tensor values to a specified min and max.
Limits the value of \(x\) to a range, whose lower limit is ‘clip_value_min’ and upper limit is ‘clip_value_max’.
\[\begin{split}out_i= \left\{ \begin{array}{align} clip\_value_{max} & \text{ if } x_i\ge clip\_value_{max} \\ x_i & \text{ if } clip\_value_{min} \lt x_i \lt clip\_value_{max} \\ clip\_value_{min} & \text{ if } x_i \le clip\_value_{min} \\ \end{array}\right.\end{split}\]Note
‘clip_value_min’ needs to be less than or equal to ‘clip_value_max’.
- Parameters
- Returns
Tensor, a clipped Tensor.
- Supported Platforms:
AscendGPU
Examples
>>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import composite as C >>> import mindspore.common.dtype as mstype >>> min_value = Tensor(5, mstype.float32) >>> max_value = Tensor(20, mstype.float32) >>> x = Tensor(np.array([[1., 25., 5., 7.], [4., 11., 6., 21.]]), mstype.float32) >>> output = C.clip_by_value(x, min_value, max_value) >>> print(output) [[ 5. 20. 5. 7.] [ 5. 11. 6. 20.]]
-
tinyms.primitives.clip_by_global_norm(x, clip_norm=1.0, use_norm=None)[source]¶ Clips tensor values by the ratio of the sum of their norms.
Note
Input ‘x’ should be a tuple or list of tensors. Otherwise, it will raise an error.
- Parameters
- Returns
tuple[Tensor], a clipped Tensor.
- Supported Platforms:
AscendGPU
Examples
>>> x1 = np.array([[2., 3.], [1., 2.]]).astype(np.float32) >>> x2 = np.array([[1., 4.], [3., 1.]]).astype(np.float32) >>> input_x = (Tensor(x1), Tensor(x2)) >>> out = clip_by_global_norm(input_x, 1.0) >>> print(out) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 2.98142403e-01, 4.47213590e-01], [ 1.49071202e-01, 2.98142403e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.49071202e-01, 5.96284807e-01], [ 4.47213590e-01, 1.49071202e-01]]))
-
tinyms.primitives.count_nonzero(x, axis=(), keep_dims=False, dtype=mindspore.int32)[source]¶ Count number of nonzero elements across axis of input tensor
- Parameters
x (Tensor) – Input data is used to count non-zero numbers.
axis (Union[int, tuple(int), list(int)]) – The dimensions to reduce. Only constant value is allowed. Default: (), reduce all dimensions.
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default: False.
dtype (Union[Number, mstype.bool_]) – The data type of the output tensor. Only constant value is allowed. Default: mstype.int32
- Returns
Tensor, number of nonzero element. The data type is dtype.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[0, 1, 0], [1, 1, 0]]).astype(np.float32)) >>> nonzero_num = count_nonzero(x=input_x, axis=[0, 1], keep_dims=True, dtype=mstype.int32) >>> print(nonzero_num) [[3]]
-
tinyms.primitives.tensor_dot(x1, x2, axes)[source]¶ Computation of Tensor contraction on arbitrary axes between tensors a and b.
Contraction allows for the summation of products of elements of a and b on specified axes. The same number of axes must be specified for both x1 and x2, and values must be within range of number of dims of both a and b.
Selected dims in both inputs must also match.
axes = 0 leads to outer product axes = 1 leads to normal matrix multiplication when inputs both 2D. axes = 1 is the same as axes = ((1,),(0,) where both a and b are 2D. axes = 2 is the same as axes = ((1,2),(0,1)) where both a and b are 3D.
- Inputs:
x1 (Tensor) - First tensor in tensor_dot with datatype float16 or float32
x2 (Tensor) - Second tensor in tensor_dot with datatype float16 or float32
axes (Union[int, tuple(int), tuple(tuple(int)), list(list(int))]) - Single value or tuple/list of length 2 with dimensions specified for a and b each. If single value N passed, automatically picks up last N dims from a input shape and first N dims from b input shape in order as axes for each respectively.
- Outputs:
Tensor, the shape of the output tensor is \((N + M)\). Where \(N\) and \(M\) are the free axes not contracted in both inputs
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x1 = Tensor(np.ones(shape=[1, 2, 3]), mindspore.float32) >>> input_x2 = Tensor(np.ones(shape=[3, 1, 2]), mindspore.float32) >>> output = C.tensor_dot(input_x1, input_x2, ((0,1),(1,2))) >>> print(output) [[2. 2. 2] [2. 2. 2] [2. 2. 2]]
-
tinyms.primitives.dot(x1, x2)[source]¶ Computation a dot product between samples in two tensors.
- Inputs:
x1 (Tensor) - First tensor in Dot op with datatype float16 or float32
x2 (Tensor) - Second tensor in Dot op with datatype float16 or float32
- Outputs:
Tensor, dot product of x1 and x2.
- Raises
TypeError – If type of x1 and x2 are not the same.
TypeError – If dtype of x1 or x2 is not float16 or float32.
ValueError – If rank of x1 or x2 less than 2.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x1 = Tensor(np.ones(shape=[2, 3]), mindspore.float32) >>> input_x2 = Tensor(np.ones(shape=[1, 3, 2]), mindspore.float32) >>> output = C.dot(input_x1, input_x2) >>> print(output) [[[3. 3.]] [[3. 3.]]]
-
tinyms.primitives.batch_dot(x1, x2, axes=None)[source]¶ Computation of batch dot product between samples in two tensors containing batch dims.
\[output = x1[batch, :] * x2[batch, :]\]- Inputs:
x1 (Tensor) - First tensor in Batch Dot op with datatype float32
x2 (Tensor) - Second tensor in Batch Dot op with datatype float32. x2’s datatype should be same as x1’s.
axes (Union[int, tuple(int), list(int)]) - Single value or tuple/list of length 2 with dimensions specified for a and b each. If single value N passed, automatically picks up last N dims from a input shape and last N dims from b input shape in order as axes for each respectively.
- Outputs:
Tensor, batch dot product of x1 and x2. The Shape of output for input shapes (batch, d1, axes, d2) and (batch, d3, axes, d4) is (batch, d1, d2, d3, d4)
- Raises
TypeError – If type of x1 and x2 are not the same.
TypeError – If dtype of x1 or x2 is not float32.
ValueError – If rank of x1 or x2 less than 2.
ValueError – If batch dim used in axes.
ValueError – If len(axes) less than 2.
ValueError – If axes is not one of those: None, int, (int, int).
ValueError – If axes reversed from negative int is too low for dimensions of input arrays.
ValueError – If axes value is too high for dimensions of input arrays.
ValueError – If batch size of x1 and x2 are not the same.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x1 = Tensor(np.ones(shape=[2, 2, 3]), mindspore.float32) >>> input_x2 = Tensor(np.ones(shape=[2, 3, 2]), mindspore.float32) >>> axes = (-1, -2) >>> output = C.batch_dot(input_x1, input_x2, axes) >>> print(output) [[[3. 3.] [3. 3.]] [[3. 3.] [3. 3.]]]
-
tinyms.primitives.repeat_elements(x, rep, axis=0)[source]¶ Repeat elements of a tensor along an axis, like np.repeat.
- Parameters
- Outputs:
One tensor with values repeated along the specified axis. If x has shape (s1, s2, …, sn) and axis is i, the output will have shape (s1, s2, …, si * rep, …, sn). The output type will be the same as the type of x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([[0, 1, 2], [3, 4, 5]]), mindspore.int32) >>> output = C.repeat_elements(x, rep = 2, axis = 0) >>> print(output) [[0 1 2] [0 1 2] [3 4 5] [3 4 5]]
-
tinyms.primitives.sequence_mask(lengths, maxlen=None)[source]¶ Returns a mask tensor representing the first N positions of each cell.
If lengths has shape [d_1, d_2, …, d_n], then the resulting tensor mask has type dtype and shape [d_1, d_2, …, d_n, maxlen], with mask[i_1, i_2, …, i_n, j] = (j < lengths[i_1, i_2, …, i_n])
- Inputs:
lengths (Tensor) - Tensor to calculate the mask for. All values in this tensor should be less than or equal to maxlen. Values greater than maxlen will be treated as maxlen. Must be type int32 or int64.
maxlen (int) - size of the last dimension of returned tensor. Must be positive and same type as elements in lengths.
- Outputs:
One mask tensor of shape lengths.shape + (maxlen,).
- Raises
- Supported Platforms:
GPU
Examples
>>> x = Tensor(np.array([[1, 3], [2, 0]])) >>> output = C.sequence_mask(x, 3) >>> print(output) [[[True, False, False], [True, True, True]], [[True, True, False], [False, False, False]]]
-
tinyms.primitives.matmul(x1, x2, dtype=None)[source]¶ Returns the matrix product of two arrays.
Note
Numpy arguments out, casting, order, subok, signature, and extobj are not supported. On GPU, the supported dtypes are np.float16 and np.float32. On CPU, the supported dtypes are np.float16 and np.float32.
- Parameters
- Returns
Tensor or scalar, the matrix product of the inputs. This is a scalar only when both x1, x2 are 1-d vectors.
- Raises
ValueError – If the last dimension of x1 is not the same size as the second-to-last dimension of x2, or if a scalar value is passed in.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x1 = Tensor(np.arange(2*3*4).reshape(2, 3, 4), mindspore.float32) >>> x2 = Tensor(np.arange(4*5).reshape(4, 5), mindspore.float32) >>> output = ops.matmul(x1, x2) >>> print(output) [[[ 70. 76. 82. 88. 94.] [ 190. 212. 234. 256. 278.] [ 310. 348. 386. 424. 462.]] [[ 430. 484. 538. 592. 646.] [ 550. 620. 690. 760. 830.] [ 670. 756. 842. 928. 1014.]]]
-
class
tinyms.primitives.ACos(*args, **kwargs)[source]¶ Computes arccosine of input tensors element-wise.
\[out_i = cos^{-1}(x_i)\]- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> acos = ops.ACos() >>> input_x = Tensor(np.array([0.74, 0.04, 0.30, 0.56]), mindspore.float32) >>> output = acos(input_x) >>> print(output) [0.7377037 1.5307858 1.2661037 0.97641146]
-
class
tinyms.primitives.Abs(*args, **kwargs)[source]¶ Returns absolute value of a tensor element-wise.
\[out_i = |x_i|\]- Inputs:
input_x (Tensor) - The input tensor. The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as the input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1.0, 1.0, 0.0]), mindspore.float32) >>> abs = ops.Abs() >>> output = abs(input_x) >>> print(output) [1. 1. 0.]
-
class
tinyms.primitives.AccumulateNV2(*args, **kwargs)[source]¶ Computes accumulation of all input tensors element-wise.
AccumulateNV2 is similar to AddN, but there is a significant difference among them: AccumulateNV2 will not wait for all of its inputs to be ready before summing. That is to say, AccumulateNV2 is able to save memory when inputs are ready at different time since the minimum temporary storage is proportional to the output size rather than the input size.
- Inputs:
input_x (Union(tuple[Tensor], list[Tensor])) - The input tuple or list is made up of multiple tensors whose dtype is number to be added together.
- Outputs:
Tensor, has the same shape and dtype as each entry of the input_x.
- Raises
TypeError – If input_x is neither tuple nor list.
- Supported Platforms:
Ascend
Examples
>>> class NetAccumulateNV2(nn.Cell): ... def __init__(self): ... super(NetAccumulateNV2, self).__init__() ... self.accumulateNV2 = ops.AccumulateNV2() ... ... def construct(self, *z): ... return self.accumulateNV2(z) ... >>> net = NetAccumulateNV2() >>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> input_y = Tensor(np.array([4, 5, 6]), mindspore.float32) >>> output = net(input_x, input_y, input_x, input_y) >>> print(output) [10. 14. 18.]
-
class
tinyms.primitives.Acosh(*args, **kwargs)[source]¶ Computes inverse hyperbolic cosine of the input element-wise.
\[out_i = cosh^{-1}(input_i)\]- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The data type should be one of the following types: float16, float32.
- Outputs:
Tensor, has the same shape and type as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> acosh = ops.Acosh() >>> input_x = Tensor(np.array([1.0, 1.5, 3.0, 100.0]), mindspore.float32) >>> output = acosh(input_x) >>> print(output) [0. 0.9624236 1.7627472 5.298292]
-
class
tinyms.primitives.Adam(*args, **kwargs)[source]¶ Updates gradients by the Adaptive Moment Estimation (Adam) algorithm.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector, \(v\) represents the 2nd moment vector, \(g\) represents gradient, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents var, \(\epsilon\) represents epsilon.
- Parameters
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
- Inputs:
var (Tensor) - Weights to be updated.
m (Tensor) - The 1st moment vector in the updating formula, has the same type as var.
v (Tensor) - the 2nd moment vector in the updating formula. Mean square gradients with the same type as var.
beta1_power (float) - \(beta_1^t\) in the updating formula.
beta2_power (float) - \(beta_2^t\) in the updating formula.
lr (float) - \(l\) in the updating formula.
beta1 (float) - The exponential decay rate for the 1st moment estimations.
beta2 (float) - The exponential decay rate for the 2nd moment estimations.
epsilon (float) - Term added to the denominator to improve numerical stability.
gradient (Tensor) - Gradient, has the same type as var.
- Outputs:
Tuple of 3 Tensor, the updated parameters.
var (Tensor) - The same shape and data type as var.
m (Tensor) - The same shape and data type as m.
v (Tensor) - The same shape and data type as v.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_adam = ops.Adam() ... self.var = Parameter(Tensor(np.ones([2, 2]).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.ones([2, 2]).astype(np.float32)), name="m") ... self.v = Parameter(Tensor(np.ones([2, 2]).astype(np.float32)), name="v") ... def construct(self, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad): ... out = self.apply_adam(self.var, self.m, self.v, beta1_power, beta2_power, lr, beta1, beta2, ... epsilon, grad) ... return out ... >>> net = Net() >>> gradient = Tensor(np.ones([2, 2]).astype(np.float32)) >>> output = net(0.9, 0.999, 0.001, 0.9, 0.999, 1e-8, gradient) >>> print(net.var.asnumpy()) [[0.9996838 0.9996838] [0.9996838 0.9996838]]
-
class
tinyms.primitives.AdamNoUpdateParam(*args, **kwargs)[source]¶ Updates gradients by Adaptive Moment Estimation (Adam) algorithm. This operator do not update the parameter, but calculate the value that should be added to the parameter instead.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ \Delta{w} = - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector, \(v\) represents the 2nd moment vector, \(g\) represents gradient, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents the parameter to be updated, \(\epsilon\).
- Parameters
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
- Inputs:
m (Tensor) - The 1st moment vector in the updating formula. The data type must be float32.
v (Tensor) - the 2nd moment vector in the updating formula. The shape must be the same as m. The data type must be float32.
beta1_power (Tensor) - \(beta_1^t\) in the updating formula. The data type must be float32.
beta2_power (Tensor) - \(beta_2^t\) in the updating formula. The data type must be float32.
lr (Tensor) - \(l\) in the updating formula. The data type must be float32.
beta1 (Tensor) - The exponential decay rate for the 1st moment estimations. The data type must be float32.
beta2 (Tensor) - The exponential decay rate for the 2nd moment estimations. The data type must be float32.
epsilon (Tensor) - Term added to the denominator to improve numerical stability. The data type must be float32.
gradient (Tensor) - Gradient, the shape must be the same as m, the data type must be float32.
- Outputs:
Tensor, whose shape and data type are the same with gradient, is a value that should be added to the parameter to be updated.
- Raises
- Supported Platforms:
CPU
Examples
>>> import numpy as np >>> import mindspore as ms >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.adam = ops.AdamNoUpdateParam() ... self.m = Parameter(Tensor(np.array([[0.1, 0.1, 0.1], [0.2, 0.2, 0.2]]).astype(np.float32)), ... name="m") ... self.v = Parameter(Tensor(np.array([[0.1, 0.1, 0.1], [0.2, 0.2, 0.2]]).astype(np.float32)), ... name="v") ... def construct(self, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad): ... out = self.adam(self.m, self.v, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad) ... return out >>> net = Net() >>> beta1_power = Tensor(0.9, ms.float32) >>> beta2_power = Tensor(0.999, ms.float32) >>> lr = Tensor(0.001, ms.float32) >>> beta1 = Tensor(0.9, ms.float32) >>> beta2 = Tensor(0.999, ms.float32) >>> epsilon = Tensor(1e-8, ms.float32) >>> gradient = Tensor(np.array([[0.1, 0.1, 0.1], [0.1, 0.1, 0.1]]).astype(np.float32)) >>> result = net(beta1_power, beta2_power, lr, beta1, beta2, epsilon, gradient) >>> print(result) [[-0.00010004 -0.00010004 -0.00010004] [-0.00013441 -0.00013441 -0.00013441]]
-
class
tinyms.primitives.Add(*args, **kwargs)[source]¶ Adds two input tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
\[out_{i} = x_{i} + y_{i}\]- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> add = ops.Add() >>> input_x = Tensor(np.array([1, 2, 3]).astype(np.float32)) >>> input_y = Tensor(np.array([4, 5, 6]).astype(np.float32)) >>> output = add(input_x, input_y) >>> print(output) [5. 7. 9.]
-
class
tinyms.primitives.AddN(*args, **kwargs)[source]¶ Computes addition of all input tensors element-wise.
All input tensors must have the same shape.
- Inputs:
input_x (Union(tuple[Tensor], list[Tensor])) - The input tuple or list is made up of multiple tensors whose dtype is number or bool to be added together.
- Outputs:
Tensor, has the same shape and dtype as each entry of the input_x.
- Raises
TypeError – If input_x is neither tuple nor list.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class NetAddN(nn.Cell): ... def __init__(self): ... super(NetAddN, self).__init__() ... self.addN = ops.AddN() ... ... def construct(self, *z): ... return self.addN(z) ... >>> net = NetAddN() >>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> input_y = Tensor(np.array([4, 5, 6]), mindspore.float32) >>> output = net(input_x, input_y, input_x, input_y) >>> print(output) [10. 14. 18.]
-
class
tinyms.primitives.AllGather(*args, **kwargs)[source]¶ Gathers tensors from the specified communication group.
Note
The tensors must have the same shape and format in all processes of the collection.
- Parameters
group (str) – The communication group to work on. Default: “hccl_world_group”.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor. If the number of devices in the group is N, then the shape of output is \((N, x_1, x_2, ..., x_R)\).
- Raises
TypeError – If group is not a str.
ValueError – If the local rank id of the calling process in the group is larger than the group’s rank size.
- Supported Platforms:
AscendGPU
Examples
>>> # This example should be run with two devices. Refer to the tutorial > Distributed Training on mindspore.cn. >>> import numpy as np >>> import mindspore.ops.operations as ops >>> import mindspore.nn as nn >>> from mindspore.communication import init >>> from mindspore import Tensor, context >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() ... class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.allgather = ops.AllGather() ... ... def construct(self, x): ... return self.allgather(x) ... >>> input_ = Tensor(np.ones([2, 8]).astype(np.float32)) >>> net = Net() >>> output = net(input_) >>> print(output) [[1. 1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1. 1.] [1. 1. 1. 1. 1. 1. 1. 1.]]
-
class
tinyms.primitives.AllReduce(*args, **kwargs)[source]¶ Reduces the tensor data across all devices in such a way that all devices will get the same final result.
Note
The operation of AllReduce does not support “prod” currently. The tensors must have the same shape and format in all processes of the collection.
- Parameters
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape of the input, i.e., \((x_1, x_2, ..., x_R)\). The contents depend on the specified operation.
- Raises
TypeError – If any of op and group is not a str, or fusion is not an integer, or the input’s dtype is bool.
ValueError – If the op is “prod”.
- Supported Platforms:
AscendGPU
Examples
>>> from mindspore.communication import init >>> from mindspore import Tensor >>> from mindspore.ops.operations.comm_ops import ReduceOp >>> import mindspore.nn as nn >>> import mindspore.ops.operations as ops >>> >>> init() >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.allreduce_sum = ops.AllReduce(ReduceOp.SUM, group="nccl_world_group") ... ... def construct(self, x): ... return self.allreduce_sum(x) ... >>> input_ = Tensor(np.ones([2, 8]).astype(np.float32)) >>> net = Net() >>> output = net(input_) >>> print(output) [[4. 5. 6. 0. 0. 0. 0. 0.] [0. 0. 0. 0. 0. 0. 0. 0.]]
-
class
tinyms.primitives.AllSwap(*args, **kwargs)[source]¶ AllSwap is a collective operation.
AllSwap sends data from the all processes to the all processes in the specified group. It has two phases:
The scatter phase: On each process, the operand is split into the send size of blocks along the 0-th axis, and the blocks are scattered to all processes, e.g., the ith block is send to the ith process.
The gather phase: Each process concatenates the received blocks along the 0-th axis.
Note
The tensors must have the same format in all processes of the collection.
- Parameters
group (str) – The communication group name.
- Inputs:
tensor_in (tensor): A 2-D tensor. On each process, divide blocks into number of the send size. send_size (tensor): A 1-D int64 tensor. The element is the send data size for each process. recv_size (tensor): A 1-D int64 tensor. The element is the receive data size for each process.
- Returns
The result tensor.
- Return type
tensor_out (tensor)
- Raises
TypeError – If group is not a string.
-
class
tinyms.primitives.AngleAtomEnergy(*args, **kwargs)[source]¶ Add the potential energy caused by angle terms to the total potential energy of each atom. Assume the number of angles is M and the number of atoms is N.
The calculation formula is the same as operator AngleEnergy().
- Parameters
angle_numbers (int32) – the number of angles M.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the 1st atom index of each angle.
atom_b (Tensor, int32) - [M,], the 2nd and the central atom index of each angle.
atom_c (Tensor, int32) - [M,], the 3rd atom index of each angle.
angle_k (Tensor, float32) - [M,], the force constant for each angle.
angle_theta0 (Tensor, float32) - [M,], the equilibrium position value for each angle.
- Outputs:
ene (Tensor, float32) - [N,], the accumulated potential energy for each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.AngleEnergy(*args, **kwargs)[source]¶ Calculate the energy caused by 3-atoms angle term. Assume the number of angles is M and the number of atoms is N.
\[dr_{ab} = (x_b-x_a, y_b-y_a, z_b-z_a)\]\[dr_{cb} = (x_b-x_c, y_b-y_c, z_b-z_c)\]\[theta = arccos(inner_product(dr_{ab}, dr_{cb})/|dr_{ab}|/|dr_{cb}|)\]\[E = k*(theta - theta_0)^2\]- Parameters
angle_numbers (int32) – the number of angles M.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the 1st atom index of each angle.
atom_b (Tensor, int32) - [M,], the 2nd and the central atom index of each angle.
atom_c (Tensor, int32) - [M,], the 3rd atom index of each angle.
angle_k (Tensor, float32) - [M,], the force constant for each angle.
angle_theta0 (Tensor, float32) - [M,], the equilibrium position value for each angle.
- Outputs:
ene (Tensor, float32) - [M,], the potential energy for each angle term.
- Supported Platforms:
GPU
-
class
tinyms.primitives.AngleForce(*args, **kwargs)[source]¶ Calculate the force exerted by angles made of 3 atoms on the corresponding atoms. Assume the number of angles is M and the number of atoms is N.
\[dr_{ab} = (x_b-x_a, y_b-y_a, z_b-z_a)\]\[dr_{cb} = (x_b-x_c, y_b-y_c, z_b-z_c)\]\[theta = arccos(inner_product(dr_{ab}, dr_{cb})/|dr_{ab}|/|dr_{cb}|)\]\[F_a = -2*k*(theta-theta_0)/sin(theta)*[cos(theta)/|dr_{ab}|^2*dr_{ab} - 1/|dr_{ab}|/|dr_{cb}|*dr_{cb}]\]\[F_c = -2*k*(theta-theta_0)/sin(theta)*[cos(theta)/|dr_{cb}|^2*dr_{cb} - 1/|dr_{cb}|/|dr_{ab}|*dr_{ab}]\]\[F_b = -F_a - F_c\]- Parameters
angle_numbers (int32) – the number of angles M.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the 1st atom index of each angle.
atom_b (Tensor, int32) - [M,], the 2nd and the central atom index of each angle.
atom_c (Tensor, int32) - [M,], the 3rd atom index of each angle.
angle_k (Tensor, float32) - [M,], the force constant for each angle.
angle_theta0 (Tensor, float32) - [M,], the equilibrium position value for each angle.
- Outputs:
frc_f (Tensor, float32) - [N, 3], the force felt by each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.AngleForceWithAtomEnergy(*args, **kwargs)[source]¶ Calculate angle force and potential energy together. Assume the number of angles is M and the number of atoms is N.
The calculation formula is the same as operator AngleForce() and AngleEnergy().
- Parameters
angle_numbers (int32) – the number of angles M.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the 1st atom index of each angle.
atom_b (Tensor, int32) - [M,], the 2nd and the central atom index of each angle.
atom_c (Tensor, int32) - [M,], the 3rd atom index of each angle.
angle_k (Tensor, float32) - [M,], the force constant for each angle.
angle_theta0 (Tensor, float32) - [M,], the equilibrium position value for each angle.
- Outputs:
frc_f (Tensor, float32) - [N, 3], same as operator AngleForce().
ene (Tensor, float) - [N,], same as operator AngleAtomEnergy().
- Supported Platforms:
GPU
-
class
tinyms.primitives.ApplyAdaMax(*args, **kwargs)[source]¶ Updates relevant entries according to the adamax scheme.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m_{t} = \beta_1 * m_{t-1} + (1 - \beta_1) * g \\ v_{t} = \max(\beta_2 * v_{t-1}, \left| g \right|) \\ var = var - \frac{l}{1 - \beta_1^t} * \frac{m_{t}}{v_{t} + \epsilon} \end{array}\end{split}\]\(t\) represents updating step while \(m\) represents the 1st moment vector, \(m_{t-1}\) is the last momentent of \(m_{t}\), \(v\) represents the 2nd moment vector, \(v_{t-1}\) is the last momentent of \(v_{t}\), \(l\) represents scaling factor lr, \(g\) represents grad, \(\beta_1, \beta_2\) represent beta1 and beta2, \(beta_1^t\) represents beta1_power, \(var\) represents the variable to be updated, \(\epsilon\) represents epsilon.
Inputs of var, m, v and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable to be updated. With float32 or float16 data type.
m (Parameter) - The 1st moment vector in the updating formula, has the same shape and type as var. With float32 or float16 data type.
v (Parameter) - The 2nd moment vector in the updating formula. Mean square gradients with the same shape and type as var. With float32 or float16 data type.
beta1_power (Union[Number, Tensor]) - \(beta_1^t\) in the updating formula, must be scalar. With float32 or float16 data type.
lr (Union[Number, Tensor]) - Learning rate, \(l\) in the updating formula, must be scalar. With float32 or float16 data type.
beta1 (Union[Number, Tensor]) - The exponential decay rate for the 1st moment estimations, must be scalar. With float32 or float16 data type.
beta2 (Union[Number, Tensor]) - The exponential decay rate for the 2nd moment estimations, must be scalar. With float32 or float16 data type.
epsilon (Union[Number, Tensor]) - A small value added for numerical stability, must be scalar. With float32 or float16 data type.
grad (Tensor) - A tensor for gradient, has the same shape and type as var. With float32 or float16 data type.
- Outputs:
Tuple of 3 Tensor, the updated parameters.
var (Tensor) - The same shape and data type as var.
m (Tensor) - The same shape and data type as m.
v (Tensor) - The same shape and data type as v.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_ada_max = ops.ApplyAdaMax() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="m") ... self.v = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="v") ... def construct(self, beta1_power, lr, beta1, beta2, epsilon, grad): ... out = self.apply_ada_max(self.var, self.m, self.v, beta1_power, lr, beta1, beta2, epsilon, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> beta1_power =Tensor(0.9, mstype.float32) >>> lr = Tensor(0.001, mstype.float32) >>> beta1 = Tensor(0.9, mstype.float32) >>> beta2 = Tensor(0.99, mstype.float32) >>> epsilon = Tensor(1e-10, mstype.float32) >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(beta1_power, lr, beta1, beta2, epsilon, grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.44221461e-01, 7.07908988e-01], [ 5.97648144e-01, 5.29388547e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 4.38093781e-01, 6.73864365e-01], [ 4.00932074e-01, 8.11308622e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 9.54026103e-01, 9.25596654e-01], [ 7.83807814e-01, 5.23605943e-01]]))
-
class
tinyms.primitives.ApplyAdadelta(*args, **kwargs)[source]¶ Updates relevant entries according to the adadelta scheme.
\[accum = \rho * accum + (1 - \rho) * grad^2\]\[\text{update} = \sqrt{\text{accum_update} + \epsilon} * \frac{grad}{\sqrt{accum + \epsilon}}\]\[\text{accum_update} = \rho * \text{accum_update} + (1 - \rho) * update^2\]\[var -= lr * update\]Inputs of var, accum, accum_update and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Weights to be updated. With float32 or float16 data type.
accum (Parameter) - Accumulation to be updated, has the same shape and type as var. With float32 or float16 data type.
accum_update (Parameter) - Accum_update to be updated, has the same shape and type as var. With float32 or float16 data type.
lr (Union[Number, Tensor]) - Learning rate, must be scalar. With float32 or float16 data type.
rho (Union[Number, Tensor]) - Decay rate, must be scalar. With float32 or float16 data type.
epsilon (Union[Number, Tensor]) - A small value added for numerical stability, must be scalar. With float32 or float16 data type.
grad (Tensor) - Gradients, has the same shape and type as var. With float32 or float16 data type.
- Outputs:
Tuple of 3 Tensor, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
accum_update (Tensor) - The same shape and data type as accum_update.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_adadelta = ops.ApplyAdadelta() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... self.accum_update = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum_update") ... def construct(self, lr, rho, epsilon, grad): ... out = self.apply_adadelta(self.var, self.accum, self.accum_update, lr, rho, epsilon, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> lr = Tensor(0.001, mstype.float32) >>> rho = Tensor(0.0, mstype.float32) >>> epsilon = Tensor(1e-6, mstype.float32) >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(lr, rho, epsilon, grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.47831833e-01, 7.14570105e-01], [ 6.01873636e-01, 5.44156015e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 3.22674602e-01, 8.56729150e-01], [ 5.04612131e-03, 7.59151531e-03]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 9.63660717e-01, 3.83442074e-01], [ 7.91569054e-01, 5.28826237e-01]]))
-
class
tinyms.primitives.ApplyAdagrad(*args, **kwargs)[source]¶ Updates relevant entries according to the adagrad scheme.
\[accum += grad * grad\]\[var -= lr * grad * \frac{1}{\sqrt{accum}}\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent.. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
update_slots (bool) – If True, accum will be updated. Default: True.
- Inputs:
var (Parameter) - Variable to be updated. With float32 or float16 data type.
accum (Parameter) - Accumulation to be updated. The shape and dtype must be the same as var. With float32 or float16 data type.
lr (Union[Number, Tensor]) - The learning rate value, must be scalar. With float32 or float16 data type.
grad (Tensor) - A tensor for gradient. The shape and dtype must be the same as var. With float32 or float16 data type.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Raises
- Supported Platforms:
AscendCPUGPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_adagrad = ops.ApplyAdagrad() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... def construct(self, lr, grad): ... out = self.apply_adagrad(self.var, self.accum, lr, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> lr = Tensor(0.001, mstype.float32) >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(lr, grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.47984838e-01, 7.14758754e-01], [ 6.01995945e-01, 5.44394553e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.35230064e+00, 7.92921484e-01], [ 1.06441569e+00, 1.17150283e+00]]))
-
class
tinyms.primitives.ApplyAdagradV2(*args, **kwargs)[source]¶ Updates relevant entries according to the adagradv2 scheme.
\[accum += grad * grad\]\[var -= lr * grad * \frac{1}{\sqrt{accum} + \epsilon}\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
- Inputs:
var (Parameter) - Variable to be updated. With float16 or float32 data type.
accum (Parameter) - Accumulation to be updated. The shape and dtype must be the same as var. With float16 or float32 data type.
lr (Union[Number, Tensor]) - The learning rate value, must be a float number or a scalar tensor with float16 or float32 data type.
grad (Tensor) - A tensor for gradient. The shape and dtype must be the same as var. With float16 or float32 data type.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as m.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_adagrad_v2 = ops.ApplyAdagradV2(epsilon=1e-6) ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... def construct(self, lr, grad): ... out = self.apply_adagrad_v2(self.var, self.accum, lr, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> lr = Tensor(0.001, mstype.float32) >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(lr, grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.47984838e-01, 7.14758754e-01], [ 6.01995945e-01, 5.44394553e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.35230064e+00, 7.92921484e-01], [ 1.06441569e+00, 1.17150283e+00]]))
-
class
tinyms.primitives.ApplyAddSign(*args, **kwargs)[source]¶ Updates relevant entries according to the AddSign algorithm.
\[\begin{split}\begin{array}{ll} \\ m_{t} = \beta * m_{t-1} + (1 - \beta) * g \\ \text{update} = (\alpha + \text{sign_decay} * sign(g) * sign(m)) * g \\ var = var - lr_{t} * \text{update} \end{array}\end{split}\]\(t\) represents updating step while \(m\) represents the 1st moment vector, \(m_{t-1}\) is the last momentent of \(m_{t}\), \(lr\) represents scaling factor lr, \(g\) represents grad.
Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable tensor to be updated. With float32 or float16 data type.
m (Parameter) - Variable tensor to be updated, has the same dtype as var.
lr (Union[Number, Tensor]) - The learning rate value, must be a scalar. With float32 or float16 data type.
alpha (Union[Number, Tensor]) - Must be a scalar. With float32 or float16 data type.
sign_decay (Union[Number, Tensor]) - Must be a scalar. With float32 or float16 data type.
beta (Union[Number, Tensor]) - The exponential decay rate, must be a scalar. With float32 or float16 data type.
grad (Tensor) - A tensor of the same type as var, for the gradient.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
m (Tensor) - The same shape and data type as m.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_add_sign = ops.ApplyAddSign() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="m") ... self.lr = 0.001 ... self.alpha = 1.0 ... self.sign_decay = 0.99 ... self.beta = 0.9 ... def construct(self, grad): ... out = self.apply_add_sign(self.var, self.m, self.lr, self.alpha, self.sign_decay, self.beta, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.46895862e-01, 7.14426279e-01], [ 6.01187825e-01, 5.43830693e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 4.77655590e-01, 6.19648814e-01], [ 4.73001003e-01, 8.55485201e-01]]))
-
class
tinyms.primitives.ApplyCenteredRMSProp(*args, **kwargs)[source]¶ Optimizer that implements the centered RMSProp algorithm. Please refer to the usage in source code of nn.RMSProp.
The updating formulas of ApplyCenteredRMSProp algorithm are as follows,
\[\begin{split}\begin{array}{ll} \\ g_{t} = \rho g_{t-1} + (1 - \rho)\nabla Q_{i}(w) \\ s_{t} = \rho s_{t-1} + (1 - \rho)(\nabla Q_{i}(w))^2 \\ m_{t} = \beta m_{t-1} + \frac{\eta} {\sqrt{s_{t} - g_{t}^2 + \epsilon}} \nabla Q_{i}(w) \\ w = w - m_{t} \end{array}\end{split}\]where \(w\) represents var, which will be updated. \(g_{t}\) represents mean_gradient, \(g_{t-1}\) is the last momentent of \(g_{t}\). \(s_{t}\) represents mean_square, \(s_{t-1}\) is the last momentent of \(s_{t}\), \(m_{t}\) represents moment, \(m_{t-1}\) is the last momentent of \(m_{t}\). \(\\rho\) represents decay. \(\\beta\) is the momentum term, represents momentum. \(\\epsilon\) is a smoothing term to avoid division by zero, represents epsilon. \(\\eta\) represents learning_rate. \(\\nabla Q_{i}(w)\) represents grad.
- Parameters
use_locking (bool) – Whether to enable a lock to protect the variable and accumlation tensors from being updated. Default: False.
- Inputs:
var (Tensor) - Weights to be update.
mean_gradient (Tensor) - Mean gradients, must have the same type as var.
mean_square (Tensor) - Mean square gradients, must have the same type as var.
moment (Tensor) - Delta of var, must have the same type as var.
grad (Tensor) - Gradient, must have the same type as var.
learning_rate (Union[Number, Tensor]) - Learning rate. Must be a float number or a scalar tensor with float16 or float32 data type.
decay (float) - Decay rate.
momentum (float) - Momentum.
epsilon (float) - Ridge term.
- Outputs:
Tensor, parameters to be update.
- Raises
TypeError – If use_locking is not a bool.
TypeError – If var, mean_gradient, mean_square, moment or grad is not a Tensor.
TypeError – If learing_rate is neither a Number nor a Tensor.
TypeError – If dtype of learing_rate is neither float16 nor float32.
TypeError – If dtype of decay, momentum or epsilon is not float.
- Supported Platforms:
AscendGPUCPU
Examples
>>> centered_rms_prop = ops.ApplyCenteredRMSProp() >>> input_x = Tensor(np.arange(-2, 2).astype(np.float32).reshape(2, 2), mindspore.float32) >>> mean_grad = Tensor(np.arange(4).astype(np.float32).reshape(2, 2), mindspore.float32) >>> mean_square = Tensor(np.arange(-3, 1).astype(np.float32).reshape(2, 2), mindspore.float32) >>> moment = Tensor(np.arange(4).astype(np.float32).reshape(2, 2), mindspore.float32) >>> grad = Tensor(np.arange(4).astype(np.float32).reshape(2, 2), mindspore.float32) >>> learning_rate = Tensor(0.9, mindspore.float32) >>> decay = 0.0 >>> momentum = 1e-10 >>> epsilon = 0.05 >>> output = centered_rms_prop(input_x, mean_grad, mean_square, moment, grad, ... learning_rate, decay, momentum, epsilon) >>> output Tensor(shape=[2, 2], dtype=Float32, value= [[-2.00000000e+00, -5.02492237e+00], [-8.04984474e+00, -1.10747662e+01]])
-
class
tinyms.primitives.ApplyFtrl(*args, **kwargs)[source]¶ Updates relevant entries according to the FTRL scheme.
- Parameters
use_locking (bool) – Use locks for updating operation if true . Default: False.
- Inputs:
var (Parameter) - The variable to be updated. The data type must be float16 or float32.
accum (Parameter) - The accumulation to be updated, must be same type and shape as var.
linear (Parameter) - the linear coefficient to be updated, must be same type and shape as var.
grad (Tensor) - Gradient. The data type must be float16 or float32.
lr (Union[Number, Tensor]) - The learning rate value, must be positive. Default: 0.001. It must be a float number or a scalar tensor with float16 or float32 data type.
l1 (Union[Number, Tensor]) - l1 regularization strength, must be greater than or equal to zero. Default: 0.0. It must be a float number or a scalar tensor with float16 or float32 data type.
l2 (Union[Number, Tensor]) - l2 regularization strength, must be greater than or equal to zero. Default: 0.0. It must be a float number or a scalar tensor with float16 or float32 data type.
lr_power (Union[Number, Tensor]) - Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero. Default: -0.5. It must be a float number or a scalar tensor with float16 or float32 data type.
- Outputs:
var (Tensor) - represents the updated var. As the input parameters has been updated in-place, this value is always zero when the platforms is GPU.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter, Tensor >>> import mindspore.context as context >>> from mindspore.ops import operations as ops >>> class ApplyFtrlNet(nn.Cell): ... def __init__(self): ... super(ApplyFtrlNet, self).__init__() ... self.apply_ftrl = ops.ApplyFtrl() ... self.lr = 0.001 ... self.l1 = 0.0 ... self.l2 = 0.0 ... self.lr_power = -0.5 ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... self.linear = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="linear") ... ... def construct(self, grad): ... out = self.apply_ftrl(self.var, self.accum, self.linear, grad, self.lr, self.l1, self.l2, ... self.lr_power) ... return out ... >>> np.random.seed(0) >>> net = ApplyFtrlNet() >>> input_x = Tensor(np.random.randint(-4, 4, (2, 2)), mindspore.float32) >>> output = net(input_x) >>> output Tensor(shape=[2, 2], dtype=Float32, value= [[ 4.61418092e-01, 5.30964255e-01], [ 2.68715084e-01, 3.82065028e-01]])
-
class
tinyms.primitives.ApplyGradientDescent(*args, **kwargs)[source]¶ Updates relevant entries according to the following.
\[var = var - \alpha * \delta\]Inputs of var and delta comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable tensor to be updated. With float32 or float16 data type.
alpha (Union[Number, Tensor]) - Scaling factor, must be a scalar. With float32 or float16 data type.
delta (Tensor) - A tensor for the change, has the same type as var.
- Outputs:
Tensor, represents the updated var.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_gradient_descent = ops.ApplyGradientDescent() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.alpha = 0.001 ... def construct(self, delta): ... out = self.apply_gradient_descent(self.var, self.alpha, delta) ... return out ... >>> net = Net() >>> delta = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(delta) >>> print(output.shape) (2, 2)
-
class
tinyms.primitives.ApplyMomentum(*args, **kwargs)[source]¶ Optimizer that implements the Momentum algorithm.
Refer to the paper On the importance of initialization and momentum in deep learning for more details.
Refer to
mindspore.nn.Momentumfor more details about the formula and usage.Inputs of variable, accumulation and gradient comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. Data type conversion of Parameter is not supported. RuntimeError exception will be thrown.
- Parameters
- Inputs:
variable (Parameter) - Weights to be updated. data type must be float.
accumulation (Parameter) - Accumulated gradient value by moment weight. Has the same data type with variable.
learning_rate (Union[Number, Tensor]) - The learning rate value, must be a float number or a scalar tensor with float data type.
gradient (Tensor) - Gradient, has the same data type as variable.
momentum (Union[Number, Tensor]) - Momentum, must be a float number or a scalar tensor with float data type.
- Outputs:
Tensor, parameters to be updated.
- Raises
TypeError – If the use_locking or use_nesterov is not a bool or gradient_scale is not a float.
- Supported Platforms:
AscendGPUCPU
Examples
Please refer to the usage in
mindspore.nn.Momentum.
-
class
tinyms.primitives.ApplyPowerSign(*args, **kwargs)[source]¶ Updates relevant entries according to the AddSign algorithm.
\[\begin{split}\begin{array}{ll} \\ m_{t} = \beta * m_{t-1} + (1 - \beta) * g \\ \text{update} = \exp(\text{logbase} * \text{sign_decay} * sign(g) * sign(m)) * g \\ var = var - lr_{t} * \text{update} \end{array}\end{split}\]\(t\) represents updating step while \(m\) represents the 1st moment vector, \(m_{t-1}\) is the last momentent of \(m_{t}\), \(lr\) represents scaling factor lr, \(g\) represents grad.
All of inputs comply with the implicit type conversion rules to make the data types consistent. If lr, logbase, sign_decay or beta is a number, the number is automatically converted to Tensor, and the data type is consistent with the Tensor data type involved in the operation. If inputs are tensors and have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable tensor to be updated. With float32 or float16 data type. If data type of var is float16, all inputs must have the same data type as var.
m (Parameter) - Variable tensor to be updated, has the same dtype as var.
lr (Union[Number, Tensor]) - The learning rate value, must be a scalar. With float32 or float16 data type.
logbase (Union[Number, Tensor]) - Must be a scalar. With float32 or float16 data type.
sign_decay (Union[Number, Tensor]) - Must be a scalar. With float32 or float16 data type.
beta (Union[Number, Tensor]) - The exponential decay rate, must be a scalar. With float32 or float16 data type.
grad (Tensor) - A tensor of the same type as var, for the gradient.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
m (Tensor) - The same shape and data type as m.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_power_sign = ops.ApplyPowerSign() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="m") ... self.lr = 0.001 ... self.logbase = np.e ... self.sign_decay = 0.99 ... self.beta = 0.9 ... def construct(self, grad): ... out = self.apply_power_sign(self.var, self.m, self.lr, self.logbase, ... self.sign_decay, self.beta, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.34601569e-01, 7.09534407e-01], [ 5.91087103e-01, 5.37083089e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 4.77655590e-01, 6.19648814e-01], [ 4.73001003e-01, 8.55485201e-01]]))
-
class
tinyms.primitives.ApplyProximalAdagrad(*args, **kwargs)[source]¶ Updates relevant entries according to the proximal adagrad algorithm.
\[accum += grad * grad\]\[\text{prox_v} = var - lr * grad * \frac{1}{\sqrt{accum}}\]\[var = \frac{sign(\text{prox_v})}{1 + lr * l2} * \max(\left| \text{prox_v} \right| - lr * l1, 0)\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – If true, the var and accumulation tensors will be protected from being updated. Default: False.
- Inputs:
var (Parameter) - Variable to be updated. The data type must be float16 or float32.
accum (Parameter) - Accumulation to be updated. Must has the same shape and dtype as var.
lr (Union[Number, Tensor]) - The learning rate value, must be scalar. The data type must be float16 or float32.
l1 (Union[Number, Tensor]) - l1 regularization strength, must be scalar. The data type must be float16 or float32.
l2 (Union[Number, Tensor]) - l2 regularization strength, must be scalar. The data type must be float16 or float32.
grad (Tensor) - Gradient with the same shape and dtype as var.
- Outputs:
Tuple of 2 Tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_proximal_adagrad = ops.ApplyProximalAdagrad() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="accum") ... self.lr = 0.01 ... self.l1 = 0.0 ... self.l2 = 0.0 ... def construct(self, grad): ... out = self.apply_proximal_adagrad(self.var, self.accum, self.lr, self.l1, self.l2, grad) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(grad) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 5.40526688e-01, 7.10883260e-01], [ 5.95089436e-01, 5.39996684e-01]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.35230064e+00, 7.92921484e-01], [ 1.06441569e+00, 1.17150283e+00]]))
-
class
tinyms.primitives.ApplyProximalGradientDescent(*args, **kwargs)[source]¶ Updates relevant entries according to the FOBOS(Forward Backward Splitting) algorithm.
\[\text{prox_v} = var - \alpha * \delta\]\[var = \frac{sign(\text{prox_v})}{1 + \alpha * l2} * \max(\left| \text{prox_v} \right| - alpha * l1, 0)\]Inputs of var and delta comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
var (Parameter) - Variable tensor to be updated. With float32 or float16 data type.
alpha (Union[Number, Tensor]) - Scaling factor, must be a scalar. With float32 or float16 data type.
l1 (Union[Number, Tensor]) - l1 regularization strength, must be scalar. With float32 or float16 data type.
l2 (Union[Number, Tensor]) - l2 regularization strength, must be scalar. With float32 or float16 data type.
delta (Tensor) - A tensor for the change, has the same type as var.
- Outputs:
Tensor, represents the updated var.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.apply_proximal_gradient_descent = ops.ApplyProximalGradientDescent() ... self.var = Parameter(Tensor(np.random.rand(2, 2).astype(np.float32)), name="var") ... self.alpha = 0.001 ... self.l1 = 0.0 ... self.l2 = 0.0 ... def construct(self, delta): ... out = self.apply_proximal_gradient_descent(self.var, self.alpha, self.l1, self.l2, delta) ... return out ... >>> net = Net() >>> delta = Tensor(np.random.rand(2, 2).astype(np.float32)) >>> output = net(delta) >>> print(output.shape) (2, 2)
-
class
tinyms.primitives.ApplyRMSProp(*args, **kwargs)[source]¶ Optimizer that implements the Root Mean Square prop(RMSProp) algorithm. Please refer to the usage in source code of nn.RMSProp.
The updating formulas of ApplyRMSProp algorithm are as follows,
\[\begin{split}\begin{array}{ll} \\ s_{t} = \rho s_{t-1} + (1 - \rho)(\nabla Q_{i}(w))^2 \\ m_{t} = \beta m_{t-1} + \frac{\eta} {\sqrt{s_{t} + \epsilon}} \nabla Q_{i}(w) \\ w = w - m_{t} \end{array}\end{split}\]where \(w\) represents var, which will be updated. \(s_{t}\) represents mean_square, \(s_{t-1}\) is the last momentent of \(s_{t}\), \(m_{t}\) represents moment, \(m_{t-1}\) is the last momentent of \(m_{t}\). \(\\rho\) represents decay. \(\\beta\) is the momentum term, represents momentum. \(\\epsilon\) is a smoothing term to avoid division by zero, represents epsilon. \(\\eta\) represents learning_rate. \(\\nabla Q_{i}(w)\) represents grad.
- Parameters
use_locking (bool) – Whether to enable a lock to protect the variable and accumlation tensors from being updated. Default: False.
- Inputs:
var (Tensor) - Weights to be update.
mean_square (Tensor) - Mean square gradients, must have the same type as var.
moment (Tensor) - Delta of var, must have the same type as var.
learning_rate (Union[Number, Tensor]) - Learning rate. Must be a float number or a scalar tensor with float16 or float32 data type.
grad (Tensor) - Gradient, must have the same type as var.
decay (float) - Decay rate. Only constant value is allowed.
momentum (float) - Momentum. Only constant value is allowed.
epsilon (float) - Ridge term. Only constant value is allowed.
- Outputs:
Tensor, parameters to be update.
- Raises
TypeError – If use_locking is not a bool.
TypeError – If var, `mean_square, moment or decay is not a Tensor.
TypeError – If learning_rate is neither a Number nor a Tensor.
TypeError – If dtype of decay, momentum or epsilon is not float.
TypeError – If dtype of learning_rate is neither float16 nor float32.
ValueError – If decay, momentum or epsilon is not a constant value.
- Supported Platforms:
AscendGPUCPU
Examples
>>> apply_rms = ops.ApplyRMSProp() >>> input_x = Tensor(1., mindspore.float32) >>> mean_square = Tensor(2., mindspore.float32) >>> moment = Tensor(1., mindspore.float32) >>> grad = Tensor(2., mindspore.float32) >>> learning_rate = Tensor(0.9, mindspore.float32) >>> decay = 0.0 >>> momentum = 1e-10 >>> epsilon = 0.001 >>> output = apply_rms(input_x, mean_square, moment, learning_rate, grad, decay, momentum, epsilon) >>> output Tensor(shape=[], dtype=Float32, value= 0.100112)
-
class
tinyms.primitives.ApproximateEqual(*args, **kwargs)[source]¶ Returns True if abs(x1-x2) is smaller than tolerance element-wise, otherwise False.
Inputs of x1 and x2 comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
tolerance (float) – The maximum deviation that two elements can be considered equal. Default: 1e-05.
- Inputs:
x1 (Tensor) - A tensor. Must be one of the following types: float32, float16.
x2 (Tensor) - A tensor of the same type and shape as ‘x1’.
- Outputs:
Tensor, the shape is the same as the shape of ‘x1’, and the data type is bool.
- Raises
TypeError – If tolerance is not a float.
- Supported Platforms:
Ascend
Examples
>>> x1 = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> x2 = Tensor(np.array([2, 4, 6]), mindspore.float32) >>> approximate_equal = ops.ApproximateEqual(2.) >>> output = approximate_equal(x1, x2) >>> print(output) [ True True False]
-
class
tinyms.primitives.ArgMaxWithValue(*args, **kwargs)[source]¶ Calculates the maximum value with the corresponding index.
Calculates the maximum value along with the given axis for the input tensor. It returns the maximum values and indices.
Note
In auto_parallel and semi_auto_parallel mode, the first output index can not be used.
- Parameters
- Inputs:
input_x (Tensor) - The input tensor, can be any dimension. Set the shape of input tensor as \((x_1, x_2, ..., x_N)\).
- Outputs:
tuple (Tensor), tuple of 2 tensors, containing the corresponding index and the maximum value of the input tensor. - index (Tensor) - The index for the maximum value of the input tensor. If keep_dims is true, the shape of output tensors is \((x_1, x_2, ..., x_{axis-1}, 1, x_{axis+1}, ..., x_N)\). Otherwise, the shape is \((x_1, x_2, ..., x_{axis-1}, x_{axis+1}, ..., x_N)\). - output_x (Tensor) - The maximum value of input tensor, with the same shape as index.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), mindspore.float32) >>> index, output = ops.ArgMaxWithValue()(input_x) >>> print(index, output) 3 0.7
-
class
tinyms.primitives.ArgMinWithValue(*args, **kwargs)[source]¶ Calculates the minimum value with corresponding index, and returns indices and values.
Calculates the minimum value along with the given axis for the input tensor. It returns the minimum values and indices.
Note
In auto_parallel and semi_auto_parallel mode, the first output index can not be used.
- Parameters
- Inputs:
input_x (Tensor) - The input tensor, can be any dimension. Set the shape of input tensor as \((x_1, x_2, ..., x_N)\).
- Outputs:
tuple (Tensor), tuple of 2 tensors, containing the corresponding index and the minimum value of the input tensor. - index (Tensor) - The index for the minimum value of the input tensor. If keep_dims is true, the shape of output tensors is \((x_1, x_2, ..., x_{axis-1}, 1, x_{axis+1}, ..., x_N)\). Otherwise, the shape is \((x_1, x_2, ..., x_{axis-1}, x_{axis+1}, ..., x_N)\). - output_x (Tensor) - The minimum value of input tensor, with the same shape as index.
- Supported Platforms:
AscendCPU
Examples
>>> input_x = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), mindspore.float32) >>> output = ops.ArgMinWithValue()(input_x) >>> print(output) (Tensor(shape=[], dtype=Int32, value= 0), Tensor(shape=[], dtype=Float32, value= 0.0))
-
class
tinyms.primitives.Argmax(*args, **kwargs)[source]¶ Returns the indices of the maximum value of a tensor across the axis.
If the shape of input tensor is \((x_1, ..., x_N)\), the shape of the output tensor will be \((x_1, ..., x_{axis-1}, x_{axis+1}, ..., x_N)\).
- Parameters
axis (int) – Axis where the Argmax operation applies to. Default: -1.
output_type (
mindspore.dtype) – An optional data type of mindspore.dtype.int32. Default: mindspore.dtype.int32.
- Inputs:
input_x (Tensor) - Input tensor.
- Outputs:
Tensor, indices of the max value of input tensor across the axis.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[1, 20, 5], [67, 8, 9], [130, 24, 15]]).astype(np.float32)) >>> output = ops.Argmax(output_type=mindspore.int32)(input_x) >>> print(output) [1 0 0]
-
class
tinyms.primitives.Argmin(*args, **kwargs)[source]¶ Returns the indices of the minimum value of a tensor across the axis.
If the shape of input tensor is \((x_1, ..., x_N)\), the shape of the output tensor is \((x_1, ..., x_{axis-1}, x_{axis+1}, ..., x_N)\).
- Parameters
axis (int) – Axis where the Argmin operation applies to. Default: -1.
output_type (
mindspore.dtype) – An optional data type of mindspore.dtype.int32. Default: mindspore.dtype.int32.
- Inputs:
input_x (Tensor) - Input tensor.
- Outputs:
Tensor, indices of the min value of input tensor across the axis.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2.0, 3.1, 1.2]), mindspore.float32) >>> index = ops.Argmin()(input_x) >>> print(index) 2
-
class
tinyms.primitives.Asin(*args, **kwargs)[source]¶ Computes arcsine of input tensors element-wise.
\[out_i = sin^{-1}(x_i)\]- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> asin = ops.Asin() >>> input_x = Tensor(np.array([0.74, 0.04, 0.30, 0.56]), mindspore.float32) >>> output = asin(input_x) >>> print(output) [0.8330927 0.04001068 0.30469266 0.59438497]
-
class
tinyms.primitives.Asinh(*args, **kwargs)[source]¶ Computes inverse hyperbolic sine of the input element-wise.
\[out_i = sinh^{-1}(input_i)\]- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The data type should be one of the following types: float16, float32.
- Outputs:
Tensor, has the same shape and type as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> asinh = ops.Asinh() >>> input_x = Tensor(np.array([-5.0, 1.5, 3.0, 100.0]), mindspore.float32) >>> output = asinh(input_x) >>> print(output) [-2.3124385 1.1947632 1.8184465 5.298342 ]
-
class
tinyms.primitives.Assert(*args, **kwargs)[source]¶ Asserts that the given condition is True. If input condition evaluates to false, print the list of tensor in data.
- Parameters
summarize (int) – Print this many entries of each tensor.
- Inputs:
condition [Union[Tensor[bool], bool]] - The condition to evaluate.
input_data (Union(tuple[Tensor], list[Tensor])) - The tensors to print out when condition is false.
- Raises
Examples
>>> class AssertDemo(nn.Cell): ... def __init__(self): ... super(AssertDemo, self).__init__() ... self.assert1 = ops.Assert(summarize=10) ... self.add = ops.Add() ... ... def construct(self, x, y): ... data = self.add(x, y) ... self.assert1(True, [data]) ... return data ...
-
class
tinyms.primitives.Assign(*args, **kwargs)[source]¶ Assigns Parameter with a value.
Inputs of variable and value comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
variable (Parameter) - The Parameter.
value (Tensor) - The value to be assigned.
- Outputs:
Tensor, has the same type as original variable.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.y = mindspore.Parameter(Tensor([1.0], mindspore.float32), name="y") ... ... def construct(self, x): ... ops.Assign()(self.y, x) ... return self.y ... >>> x = Tensor([2.0], mindspore.float32) >>> net = Net() >>> output = net(x) >>> print(output) Parameter (name=y, shape=(1,), dtype=Float32, requires_grad=True)
-
class
tinyms.primitives.AssignAdd(*args, **kwargs)[source]¶ Updates a Parameter by adding a value to it.
Inputs of variable and value comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. If value is a number, the number is automatically converted to Tensor, and the data type is consistent with the Tensor data type involved in the operation. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
variable (Parameter) - The Parameter.
value (Union[numbers.Number, Tensor]) - The value to be added to the variable. It must have the same shape as variable if it is a Tensor.
- Raises
TypeError – If value is neither Number nor Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.AssignAdd = ops.AssignAdd() ... self.variable = mindspore.Parameter(initializer(1, [1], mindspore.int64), name="global_step") ... ... def construct(self, x): ... self.AssignAdd(self.variable, x) ... return self.variable ... >>> net = Net() >>> value = Tensor(np.ones([1]).astype(np.int64)*100) >>> output = net(value) >>> print(output) Parameter (name=global_step, shape=(1,), dtype=Int64, requires_grad=True)
-
class
tinyms.primitives.AssignSub(*args, **kwargs)[source]¶ Updates a Parameter by subtracting a value from it.
Inputs of variable and value comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. If value is a number, the number is automatically converted to Tensor, and the data type is consistent with the Tensor data type involved in the operation. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
variable (Parameter) - The Parameter.
value (Union[numbers.Number, Tensor]) - The value to be subtracted from the variable. It must have the same shape as variable if it is a Tensor.
- Raises
TypeError – If value is neither Number nor Tensor.
- Supported Platforms:
Ascend
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.AssignSub = ops.AssignSub() ... self.variable = mindspore.Parameter(initializer(1, [1], mindspore.int32), name="global_step") ... ... def construct(self, x): ... self.AssignSub(self.variable, x) ... return self.variable ... >>> net = Net() >>> value = Tensor(np.ones([1]).astype(np.int32)*100) >>> output = net(value) >>> print(output) Parameter (name=global_step, shape=(1,), dtype=Int32, requires_grad=True)
-
class
tinyms.primitives.Atan(*args, **kwargs)[source]¶ Computes the trigonometric inverse tangent of the input element-wise.
\[out_i = tan^{-1}(x_i)\]- Inputs:
input_x (Tensor): The input tensor. The data type should be one of the following types: float16, float32.
- Outputs:
A Tensor, has the same type as the input.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 0.0]), mindspore.float32) >>> atan = ops.Atan() >>> output = atan(input_x) >>> print(output) [0.7853982 0. ]
-
class
tinyms.primitives.Atan2(*args, **kwargs)[source]¶ Returns arctangent of input_x/input_y element-wise.
It returns \(\theta\ \in\ [-\pi, \pi]\) such that \(x = r*\sin(\theta), y = r*\cos(\theta)\), where \(r = \sqrt{x^2 + y^2}\).
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x (Tensor) - The input tensor.
input_y (Tensor) - The input tensor.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is same as input_x.
- Raises
TypeError – If input_x or input_y is not a Tensor.
- Supported Platforms:
AscendCPU
Examples
>>> input_x = Tensor(np.array([0, 1]), mindspore.float32) >>> input_y = Tensor(np.array([1, 1]), mindspore.float32) >>> atan2 = ops.Atan2() >>> output = atan2(input_x, input_y) >>> print(output) [0. 0.7853982]
-
class
tinyms.primitives.Atanh(*args, **kwargs)[source]¶ Computes inverse hyperbolic tangent of the input element-wise.
- Inputs:
input_x (Tensor): The input tensor.
- Outputs:
A Tensor, has the same type as the input.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendCPU
Examples
>>> input_x = Tensor(np.array([1.047, 0.785]), mindspore.float32) >>> atanh = ops.Atanh() >>> output = atanh(input_x) >>> print(output) [1.8869909 1.058268 ]
-
class
tinyms.primitives.AvgPool(*args, **kwargs)[source]¶ Average pooling operation.
Applies a 2D average pooling over an input Tensor which can be regarded as a composition of 2D input planes. Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), AvgPool outputs regional average in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \frac{1}{h_{ker} * w_{ker}} \sum_{m=0}^{h_{ker}-1} \sum_{n=0}^{w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the average value, is an int number that represents height and width are both kernel_size, or a tuple of two int numbers that represent height and width respectively. Default: 1.
strides (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
data_format (str) – default is ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, with shape \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If kernel_size or strides is neither int nor tuple.
ValueError – If pad_mode is neither ‘valid’ nor ‘same’ with not case sensitive.
ValueError – If data_format is neither ‘NCHW’ nor ‘NHWC’.
ValueError – If kernel_size or strides is less than 1.
ValueError – If length of shape of input is not equal to 4.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.avgpool_op = ops.AvgPool(pad_mode="VALID", kernel_size=2, strides=1) ... ... def construct(self, x): ... result = self.avgpool_op(x) ... return result ... >>> input_x = Tensor(np.arange(1 * 3 * 3 * 4).reshape(1, 3, 3, 4), mindspore.float32) >>> net = Net() >>> output = net(input_x) >>> print(output) [[[[ 2.5 3.5 4.5] [ 6.5 7.5 8.5]] [[14.5 15.5 16.5] [18.5 19.5 20.5]] [[26.5 27.5 28.5] [30.5 31.5 32.5]]]]
-
class
tinyms.primitives.BCEWithLogitsLoss(*args, **kwargs)[source]¶ Adds sigmoid activation function to input predict, and uses the given logits to compute binary cross entropy between the target and the output.
Sets input predict as X, input target as Y, output as L. Then,
\[p_{ij} = sigmoid(X_{ij}) = \frac{1}{1 + e^{-X_{ij}}}\]\[L_{ij} = -[Y_{ij} * log(p_{ij}) + (1 - Y_{ij})log(1 - p_{ij})]\]Then,
\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]- Parameters
reduction (str) – Type of reduction to be applied to loss. The optional values are ‘mean’, ‘sum’, and ‘none’. If ‘none’, do not perform reduction. Default:’mean’.
- Inputs:
predict (Tensor) - Input logits. Data type must be float16 or float32.
target (Tensor) - Ground truth label. Has the same shape with predict. Data type must be float16 or float32.
weight (Tensor) - A rescaling weight applied to the loss of each batch element. It must can be broadcast to a tensor with shape of predict. Data type must be float16 or float32.
pos_weight (Tensor) - A weight of positive examples. Must be a vector with length equal to the number of classes. It must can be broadcast to a tensor with shape of predict. Data type must be float16 or float32.
- Outputs:
Scalar. If reduction is ‘none’, it’s a tensor with the same shape and type as input predict.
- Raises
TypeError – If data type of any input is neither float16 nor float32.
ValueError – If weight or pos_weight can not be broadcast to a tensor with shape of predict.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
- Supported Platforms:
Ascend
Examples
>>> predict = Tensor(np.array([[-0.8, 1.2, 0.7], [-0.1, -0.4, 0.7]]).astype(np.float32)) >>> target = Tensor(np.array([[0.3, 0.8, 1.2], [-0.6, 0.1, 2.2]]).astype(np.float32)) >>> weight = Tensor(np.array([1.0, 1.0, 1.0]).astype(np.float32)) >>> pos_weight = Tensor(np.array([1.0, 1.0, 1.0]).astype(np.float32)) >>> loss = ops.BCEWithLogitsLoss() >>> output = loss(predict, target, weight, pos_weight) >>> print(output) 0.3463612
-
class
tinyms.primitives.BNTrainingReduce(*args, **kwargs)[source]¶ For the BatchNorm operation this operator update the moving averages for training and is used in conjunction with BNTrainingUpdate.
- Inputs:
x (Tensor) - A 4-D Tensor with float16 or float32 data type. Tensor of shape \((N, C, A, B)\).
- Outputs:
sum (Tensor) - A 1-D Tensor with float32 data type. Tensor of shape \((C,)\).
square_sum (Tensor) - A 1-D Tensor with float32 data type. Tensor of shape \((C,)\).
- Raises
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.ones([128, 3, 32, 3]), mindspore.float32) >>> bn_training_reduce = ops.BNTrainingReduce() >>> output = bn_training_reduce(input_x) >>> print(output) (Tensor(shape=[3], dtype=Float32, value= [ 1.22880000e+04, 1.22880000e+04, 1.22880000e+04]), Tensor(shape=[3], dtype=Float32, value= [ 1.22880000e+04, 1.22880000e+04, 1.22880000e+04]))
-
class
tinyms.primitives.BNTrainingUpdate(*args, **kwargs)[source]¶ For the BatchNorm operation, this operator update the moving averages for training and is used in conjunction with BNTrainingReduce.
- Parameters
- Inputs:
x (Tensor) - A 4-D Tensor with float16 or float32 data type. Tensor of shape \((N, C, A, B)\).
sum (Tensor) - A 1-D Tensor with float16 or float32 data type for the output of operator BNTrainingReduce. Tensor of shape \((C,)\).
square_sum (Tensor) - A 1-D Tensor with float16 or float32 data type for the output of operator BNTrainingReduce. Tensor of shape \((C,)\).
scale (Tensor) - A 1-D Tensor with float16 or float32, for the scaling factor. Tensor of shape \((C,)\).
offset (Tensor) - A 1-D Tensor with float16 or float32, for the scaling offset. Tensor of shape \((C,)\).
mean (Tensor) - A 1-D Tensor with float16 or float32, for the scaling mean. Tensor of shape \((C,)\).
variance (Tensor) - A 1-D Tensor with float16 or float32, for the update variance. Tensor of shape \((C,)\).
- Outputs:
y (Tensor) - Tensor, has the same shape data type as x.
mean (Tensor) - Tensor for the updated mean, with float32 data type. Has the same shape as variance.
variance (Tensor) - Tensor for the updated variance, with float32 data type. Has the same shape as variance.
batch_mean (Tensor) - Tensor for the mean of x, with float32 data type. Has the same shape as variance.
batch_variance (Tensor) - Tensor for the mean of variance, with float32 data type. Has the same shape as variance.
- Raises
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.ones([1, 2, 2, 2]), mindspore.float32) >>> sum = Tensor(np.ones([2]), mindspore.float32) >>> square_sum = Tensor(np.ones([2]), mindspore.float32) >>> scale = Tensor(np.ones([2]), mindspore.float32) >>> offset = Tensor(np.ones([2]), mindspore.float32) >>> mean = Tensor(np.ones([2]), mindspore.float32) >>> variance = Tensor(np.ones([2]), mindspore.float32) >>> bn_training_update = ops.BNTrainingUpdate() >>> output = bn_training_update(input_x, sum, square_sum, scale, offset, mean, variance) >>> print(output) (Tensor(shape=[1, 2, 2, 2], dtype=Float32, value= [[[[ 2.73200464e+00, 2.73200464e+00], [ 2.73200464e+00, 2.73200464e+00]], [[ 2.73200464e+00, 2.73200464e+00], [ 2.73200464e+00, 2.73200464e+00]]]]), Tensor(shape=[2], dtype=Float32, value= [9.24999952e-0.1, 9.24999952e-0.1]), Tensor(shape=[2], dtype=Float32, value= [ 9.24999952e-0.1, 9.24999952e-0.1]), Tensor(shape=[2], dtype=Float32, value= [ 2.50000000e-0.1, 2.50000000e-0.1]), Tensor(shape=[2], dtype=Float32, value= [ 1.87500000e-0.1, 1.87500000e-0.1]))
-
class
tinyms.primitives.BasicLSTMCell(*args, **kwargs)[source]¶ It’s similar to operator DynamicRNN. BasicLSTMCell will be deprecated in the future. Please use DynamicRNN instead.
Applies the long short-term memory (LSTM) to the input.
\[\begin{split}\begin{array}{ll} \\ i_t = \sigma(W_{ix} x_t + b_{ix} + W_{ih} h_{(t-1)} + b_{ih}) \\ f_t = \sigma(W_{fx} x_t + b_{fx} + W_{fh} h_{(t-1)} + b_{fh}) \\ \tilde{c}_t = \tanh(W_{cx} x_t + b_{cx} + W_{ch} h_{(t-1)} + b_{ch}) \\ o_t = \sigma(W_{ox} x_t + b_{ox} + W_{oh} h_{(t-1)} + b_{oh}) \\ c_t = f_t * c_{(t-1)} + i_t * \tilde{c}_t \\ h_t = o_t * \tanh(c_t) \\ \end{array}\end{split}\]Here \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product. \(W, b\) are learnable weights between the output and the input in the formula. For instance, \(W_{ix}, b_{ix}\) are the weight and bias used to transform from input \(x\) to \(i\). Details can be found in paper LONG SHORT-TERM MEMORY and Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling.
- Parameters
keep_prob (float) – If not 1.0, append Dropout layer on the outputs of each LSTM layer except the last layer. Default 1.0. The range of dropout is [0.0, 1.0].
forget_bias (float) – Add forget bias to forget gate biases in order to decrease former scale. Default: 1.0.
state_is_tuple (bool) – If true, the state is a tuple of 2 tensors, containing h and c; If false, the state is a tensor and it needs to be split first. Default: True.
activation (str) – Activation. Default: “tanh”. Only “tanh” is currently supported.
- Inputs:
x (Tensor) - Current words. Tensor of shape (batch_size, input_size). The data type must be float16 or float32.
h (Tensor) - Hidden state last moment. Tensor of shape (batch_size, hidden_size). The data type must be float16 or float32.
c (Tensor) - Cell state last moment. Tensor of shape (batch_size, hidden_size). The data type must be float16 or float32.
w (Tensor) - Weight. Tensor of shape (input_size + hidden_size, 4 x hidden_size). The data type must be float16 or float32.
b (Tensor) - Bias. Tensor of shape (4 x hidden_size). The data type must be the same as c.
- Outputs:
ct (Tensor) - Forward \(c_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
ht (Tensor) - Cell output. Tensor of shape (batch_size, hidden_size). With data type of float16.
it (Tensor) - Forward \(i_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
jt (Tensor) - Forward \(j_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
ft (Tensor) - Forward \(f_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
ot (Tensor) - Forward \(o_t\) cache at moment t. Tensor of shape (batch_size, hidden_size). Has the same type with input c.
tanhct (Tensor) - Forward \(tanh c_t\) cache at moment t. Tensor of shape (batch_size, hidden_size), has the same type with input c.
- Raises
- Supported Platforms:
Deprecated
Examples
>>> np.random.seed(0) >>> x = Tensor(np.random.rand(1, 32).astype(np.float16)) >>> h = Tensor(np.random.rand(1, 2).astype(np.float16)) >>> c = Tensor(np.random.rand(1, 2).astype(np.float16)) >>> w = Tensor(np.random.rand(34, 8).astype(np.float16)) >>> b = Tensor(np.random.rand(8, ).astype(np.float16)) >>> lstm = ops.BasicLSTMCell(keep_prob=1.0, forget_bias=1.0, state_is_tuple=True, activation='tanh') >>> output = lstm(x, h, c, w, b) >>> print(output) (Tensor(shape=[1, 2], dtype=Float16, value= ([[7.6953e-01, 9.2432e-01]]), Tensor(shape=[1, 2], dtype=Float16, value= [[1.0000e+00, 1.0000e+00]]), Tensor(shape=[1, 2], dtype=Float16, value= [[1.0000e+00, 1.0000e+00]]), Tensor(shape=[1, 2], dtype=Float16, value= [[1.0000e+00, 1.0000e+00]]), Tensor(shape=[1, 2], dtype=Float16, value= [[1.0000e+00, 1.0000e+00]]), Tensor(shape=[1, 2], dtype=Float16, value= [[7.6953e-01, 9.2432e-01]]), Tensor(shape=[1, 2], dtype=Float16, value= [[0.0000e+00, 0.0000e+00]]))
-
class
tinyms.primitives.BatchMatMul(*args, **kwargs)[source]¶ Computes matrix multiplication between two tensors by batch.
\[\text{output}[..., :, :] = \text{matrix}(a[..., :, :]) * \text{matrix}(b[..., :, :])\]The two input tensors must have the same rank and the rank must be not less than 3.
- Parameters
- Inputs:
input_x (Tensor) - The first tensor to be multiplied. The shape of the tensor is \((*B, N, C)\), where \(*B\) represents the batch size which can be multidimensional, \(N\) and \(C\) are the size of the last two dimensions. If transpose_a is True, its shape must be \((*B, C, N)\).
input_y (Tensor) - The second tensor to be multiplied. The shape of the tensor is \((*B, C, M)\). If transpose_b is True, its shape must be \((*B, M, C)\).
- Outputs:
Tensor, the shape of the output tensor is \((*B, N, M)\).
- Raises
TypeError – If transpose_a or transpose_b is not a bool.
ValueError – If length of shape of input_x is not equal to length of shape of input_y or length of shape of input_x is less than 3.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.ones(shape=[2, 4, 1, 3]), mindspore.float32) >>> input_y = Tensor(np.ones(shape=[2, 4, 3, 4]), mindspore.float32) >>> batmatmul = ops.BatchMatMul() >>> output = batmatmul(input_x, input_y) >>> print(output) [[[[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]]] [[[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]]]] >>> input_x = Tensor(np.ones(shape=[2, 4, 3, 1]), mindspore.float32) >>> input_y = Tensor(np.ones(shape=[2, 4, 3, 4]), mindspore.float32) >>> batmatmul = ops.BatchMatMul(transpose_a=True) >>> output = batmatmul(input_x, input_y) >>> print(output) [[[[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]]] [[[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]] [[3. 3. 3. 3.]]]]
-
class
tinyms.primitives.BatchNorm(*args, **kwargs)[source]¶ Batch Normalization for input data and updated parameters.
Batch Normalization is widely used in convolutional neural networks. This operation applies Batch Normalization over input to avoid internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the features using a mini-batch of data and the learned parameters which can be described in the following formula,
\[y = \frac{x - mean}{\sqrt{variance + \epsilon}} * \gamma + \beta\]where \(\gamma\) is scale, \(\beta\) is bias, \(\epsilon\) is epsilon.
- Parameters
is_training (bool) – If is_training is True, mean and variance are computed during training. If is_training is False, they’re loaded from checkpoint during inference. Default: False.
epsilon (float) – A small value added for numerical stability. Default: 1e-5.
momentum (float) – The hyper parameter to compute moving average for running_mean and running_var (e.g. \(new\_running\_mean = (1 - momentum) * running\_mean + momentum * current\_mean\)). Momentum value must be [0, 1]. Default: 0.1.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: “NCHW”.
- Inputs:
If is_training is False, inputs are Tensors.
input_x (Tensor) - Tensor of shape \((N, C)\), with float16 or float32 data type.
scale (Tensor) - Tensor of shape \((C,)\), with float16 or float32 data type.
bias (Tensor) - Tensor of shape \((C,)\), has the same data type with scale.
mean (Tensor) - Tensor of shape \((C,)\), with float16 or float32 data type.
variance (Tensor) - Tensor of shape \((C,)\), has the same data type with mean.
If is_training is True, scale, bias, mean and variance are Parameters.
input_x (Tensor) - Tensor of shape \((N, C)\), with float16 or float32 data type.
scale (Parameter) - Parameter of shape \((C,)\), with float16 or float32 data type.
bias (Parameter) - Parameter of shape \((C,)\), has the same data type with scale.
mean (Parameter) - Parameter of shape \((C,)\), with float16 or float32 data type.
variance (Parameter) - Parameter of shape \((C,)\), has the same data type with mean.
- Outputs:
Tuple of 5 Tensor, the normalized inputs and the updated parameters.
output_x (Tensor) - The same type and shape as the input_x. The shape is \((N, C)\).
updated_scale (Tensor) - Tensor of shape \((C,)\).
updated_bias (Tensor) - Tensor of shape \((C,)\).
reserve_space_1 (Tensor) - Tensor of shape \((C,)\).
reserve_space_2 (Tensor) - Tensor of shape \((C,)\).
- Raises
- Supported Platforms:
AscendCPU
Examples
>>> input_x = Tensor(np.ones([2, 2]), mindspore.float32) >>> scale = Tensor(np.ones([2]), mindspore.float32) >>> bias = Tensor(np.ones([2]), mindspore.float32) >>> mean = Tensor(np.ones([2]), mindspore.float32) >>> variance = Tensor(np.ones([2]), mindspore.float32) >>> batch_norm = ops.BatchNorm() >>> output = batch_norm(input_x, scale, bias, mean, variance) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.00000000e+00, 1.00000000e+00], [ 1.00000000e+00, 1.00000000e+00]]), Tensor(shape=[2], dtype=Float32, value= [ 1.00000000e+00, 1.00000000e+00]), Tensor(shape=[2], dtype=Float32, value= [ 1.00000000e+00, 1.00000000e+00]), Tensor(shape=[2], dtype=Float32, value= [ 1.00000000e+00, 1.00000000e+00]), Tensor(shape=[2], dtype=Float32, value= [ 1.00000000e+00, 1.00000000e+00]))
-
class
tinyms.primitives.BatchToSpace(*args, **kwargs)[source]¶ Divides batch dimension with blocks and interleaves these blocks back into spatial dimensions.
This operation will divide batch dimension N into blocks with block_size, the output tensor’s N dimension is the corresponding number of blocks after division. The output tensor’s H, W dimension is product of original H, W dimension and block_size with given amount to crop from dimension, respectively.
- Parameters
block_size (int) – The block size of division, has the value not less than 2.
crops (Union[list(int), tuple(int)]) – The crop value for H and W dimension, containing 2 subtraction lists. Each list contains 2 integers. All values must be not less than 0. crops[i] specifies the crop values for the spatial dimension i, which corresponds to the input dimension i+2. It is required that input_shape[i+2]*block_size >= crops[i][0]+crops[i][1].
- Inputs:
input_x (Tensor) - The input tensor. It must be a 4-D tensor, dimension 0 must be divisible by product of block_shape.
- Outputs:
Tensor, the output tensor with the same type as input. Assume input shape is (n, c, h, w) with block_size and crops. The output shape will be (n’, c’, h’, w’), where
\(n' = n//(block\_size*block\_size)\)
\(c' = c\)
\(h' = h*block\_size-crops[0][0]-crops[0][1]\)
\(w' = w*block\_size-crops[1][0]-crops[1][1]\)
- Raises
TypeError – If block_size or element of crops is not an int.
TypeError – If crops is neither list nor tuple.
ValueError – If block_size is less than 2.
- Supported Platforms:
Ascend
Examples
>>> block_size = 2 >>> crops = [[0, 0], [0, 0]] >>> batch_to_space = ops.BatchToSpace(block_size, crops) >>> input_x = Tensor(np.array([[[[1]]], [[[2]]], [[[3]]], [[[4]]]]), mindspore.float32) >>> output = batch_to_space(input_x) >>> print(output) [[[[1. 2.] [3. 4.]]]]
-
class
tinyms.primitives.BatchToSpaceND(*args, **kwargs)[source]¶ Divides batch dimension with blocks and interleaves these blocks back into spatial dimensions.
This operation will divide batch dimension N into blocks with block_shape, the output tensor’s N dimension is the corresponding number of blocks after division. The output tensor’s H, W dimension is product of original H, W dimension and block_shape with given amount to crop from dimension, respectively.
- Parameters
block_shape (Union[list(int), tuple(int), int]) – The block shape of dividing block with all value greater than 1. If block_shape is a tuple or list, the length of block_shape is M corresponding to the number of spatial dimensions. If block_shape is a int, the block size of M dimendions are the same, equal to block_shape. M must be 2.
crops (Union[list(int), tuple(int)]) – The crop value for H and W dimension, containing 2 subtraction list, each containing 2 int value. All values must be >= 0. crops[i] specifies the crop values for spatial dimension i, which corresponds to input dimension i+2. It is required that input_shape[i+2]*block_shape[i] > crops[i][0]+crops[i][1].
- Inputs:
input_x (Tensor) - The input tensor. It must be a 4-D tensor, dimension 0 must be divisible by product of block_shape.
- Outputs:
Tensor, the output tensor with the same type as input. Assume input shape is (n, c, h, w) with block_shape and crops. The output shape will be (n’, c’, h’, w’), where
\(n' = n//(block\_shape[0]*block\_shape[1])\)
\(c' = c\)
\(h' = h*block\_shape[0]-crops[0][0]-crops[0][1]\)
\(w' = w*block\_shape[1]-crops[1][0]-crops[1][1]\)
- Raises
TypeError – If block_shape is not one of list, tuple, int.
TypeError – If crops is neither list nor tuple.
ValueError – If length of block_shape or crops is not equal to 2.
- Supported Platforms:
Ascend
Examples
>>> block_shape = [2, 2] >>> crops = [[0, 0], [0, 0]] >>> batch_to_space_nd = ops.BatchToSpaceND(block_shape, crops) >>> input_x = Tensor(np.array([[[[1]]], [[[2]]], [[[3]]], [[[4]]]]), mindspore.float32) >>> output = batch_to_space_nd(input_x) >>> print(output) [[[[1. 2.] [3. 4.]]]]
-
class
tinyms.primitives.BesselI0e(*args, **kwargs)[source]¶ Computes BesselI0e of input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). Data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
Ascend
Examples
>>> bessel_i0e = ops.BesselI0e() >>> input_x = Tensor(np.array([0.24, 0.83, 0.31, 0.09]), mindspore.float32) >>> output = bessel_i0e(input_x) >>> print(output) [0.7979961 0.5144438 0.75117415 0.9157829 ]
-
class
tinyms.primitives.BesselI1e(*args, **kwargs)[source]¶ Computes BesselI1e of input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). Data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
Ascend
Examples
>>> bessel_i1e = ops.BesselI1e() >>> input_x = Tensor(np.array([0.24, 0.83, 0.31, 0.09]), mindspore.float32) >>> output = bessel_i1e(input_x) >>> print(output) [0.09507662 0.19699717 0.11505538 0.04116856]
-
class
tinyms.primitives.BiasAdd(*args, **kwargs)[source]¶ Returns sum of input and bias tensor.
Adds the 1-D bias tensor to the input tensor, and broadcasts the shape on all axis except for the channel axis.
- Parameters
data_format (str) – The format of input and output data. It should be ‘NHWC’, ‘NCHW’ or ‘NCDHW’， default is ‘NCHW’.
- Inputs:
input_x (Tensor) - The input tensor. The shape can be 2-5 dimensions.
bias (Tensor) - The bias tensor, with shape \((C)\). The shape of bias must be the same as input_x’s channel dimension.
- Outputs:
Tensor, with the same shape and type as input_x.
- Raises
TypeError – If data_format, input_x or bias is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.arange(6).reshape((2, 3)), mindspore.float32) >>> bias = Tensor(np.random.random(3).reshape((3,)), mindspore.float32) >>> bias_add = ops.BiasAdd() >>> output = bias_add(input_x, bias) >>> print(output.shape) (2, 3)
-
class
tinyms.primitives.BinaryCrossEntropy(*args, **kwargs)[source]¶ Computes the binary cross entropy between the target and the output.
Sets input as \(x\), input label as \(y\), output as \(\ell(x, y)\). Let,
\[L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log x_n + (1 - y_n) \cdot \log (1 - x_n) \right]\]Then,
\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]- Parameters
reduction (str) – Specifies the reduction to be applied to the output. Its value must be one of ‘none’, ‘mean’, ‘sum’. Default: ‘mean’.
- Inputs:
input_x (Tensor) - The input Tensor. The data type must be float16 or float32.
input_y (Tensor) - The label Tensor which has same shape and data type as input_x.
weight (Tensor, optional) - A rescaling weight applied to the loss of each batch element. And it must have same shape and data type as input_x. Default: None.
- Outputs:
Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as input_x. Otherwise, the output is a scalar.
- Raises
TypeError – If dtype of input_x, input_y or weight (if given) is neither float16 not float32.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
ValueError – If shape of input_y is not the same as input_x or weight (if given).
TypeError – If input_x, input_y or weight is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.binary_cross_entropy = ops.BinaryCrossEntropy() ... def construct(self, x, y, weight): ... result = self.binary_cross_entropy(x, y, weight) ... return result ... >>> net = Net() >>> input_x = Tensor(np.array([0.2, 0.7, 0.1]), mindspore.float32) >>> input_y = Tensor(np.array([0., 1., 0.]), mindspore.float32) >>> weight = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = net(input_x, input_y, weight) >>> print(output) 0.38240486
-
class
tinyms.primitives.BitwiseAnd(*args, **kwargs)[source]¶ Returns bitwise and of two tensors element-wise.
Inputs of input_x1 and input_x2 comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x1 (Tensor) - The input tensor with int16, int32 or uint16 data type.
input_x2 (Tensor) - The input tensor with same type as the input_x1.
- Outputs:
Tensor, has the same type as the input_x1.
- Raises
TypeError – If input_x1 or input_x2 is not a Tensor.
- Supported Platforms:
Ascend
Examples
>>> input_x1 = Tensor(np.array([0, 0, 1, -1, 1, 1, 1]), mindspore.int16) >>> input_x2 = Tensor(np.array([0, 1, 1, -1, -1, 2, 3]), mindspore.int16) >>> bitwise_and = ops.BitwiseAnd() >>> output = bitwise_and(input_x1, input_x2) >>> print(output) [ 0 0 1 -1 1 0 1]
-
class
tinyms.primitives.BitwiseOr(*args, **kwargs)[source]¶ Returns bitwise or of two tensors element-wise.
Inputs of input_x1 and input_x2 comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x1 (Tensor) - The input tensor with int16, int32 or uint16 data type.
input_x2 (Tensor) - The input tensor with same type as the input_x1.
- Outputs:
Tensor, has the same type as the input_x1.
- Raises
TypeError – If input_x1 or input_x2 is not a Tensor.
- Supported Platforms:
Ascend
Examples
>>> input_x1 = Tensor(np.array([0, 0, 1, -1, 1, 1, 1]), mindspore.int16) >>> input_x2 = Tensor(np.array([0, 1, 1, -1, -1, 2, 3]), mindspore.int16) >>> bitwise_or = ops.BitwiseOr() >>> output = bitwise_or(input_x1, input_x2) >>> print(output) [ 0 1 1 -1 -1 3 3]
-
class
tinyms.primitives.BitwiseXor(*args, **kwargs)[source]¶ Returns bitwise xor of two tensors element-wise.
Inputs of input_x1 and input_x2 comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x1 (Tensor) - The input tensor with int16, int32 or uint16 data type.
input_x2 (Tensor) - The input tensor with same type as the input_x1.
- Outputs:
Tensor, has the same type as the input_x1.
- Raises
TypeError – If input_x1 or input_x2 is not a Tensor.
- Supported Platforms:
Ascend
Examples
>>> input_x1 = Tensor(np.array([0, 0, 1, -1, 1, 1, 1]), mindspore.int16) >>> input_x2 = Tensor(np.array([0, 1, 1, -1, -1, 2, 3]), mindspore.int16) >>> bitwise_xor = ops.BitwiseXor() >>> output = bitwise_xor(input_x1, input_x2) >>> print(output) [ 0 1 0 0 -2 3 2]
-
class
tinyms.primitives.BondAtomEnergy(*args, **kwargs)[source]¶ Add the potential energy caused by simple harmonic bonds to the total potential energy of each atom.
The calculation formula is the same as operator BondEnergy().
- Parameters
atom_numbers (int32) – the number of atoms N.
bond_numbers (int32) – the number of harmonic bonds M.
- Inputs:
uint_crd_f (Tensor, uint32 ) - [N, 3], the unsigned int coordinate value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor (x, y, z), between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the first atom index of each bond.
atom_b (Tensor, int32) - [M,], the second atom index of each bond.
bond_k (Tensor, float32) - [M,], the force constant of each bond.
bond_r0 (Tensor, float32) - [M,], the equlibrium length of each bond.
- Outputs:
atom_ene (Tensor, float32) - [N,], the accumulated potential energy for each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.BondEnergy(*args, **kwargs)[source]¶ Calculate the harmonic potential energy between each bonded atom pair. Assume our system has N atoms and M harmonic bonds.
\[dr = (x_1-x_2, y_1-y_2, z_1-z_2)\]\[E = k*(|dr| - r_0)^2\]- Parameters
atom_numbers (int32) – the number of atoms N.
bond_numbers (int32) – the number of harmonic bonds M.
- Inputs:
uint_crd_f (Tensor, uint32 ) - [N, 3], the unsigned int coordinate value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor (x, y, z), between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the first atom index of each bond.
atom_b (Tensor, int32) - [M,], the second atom index of each bond.
bond_k (Tensor, float32) - [M,], the force constant of each bond.
bond_r0 (Tensor, float32) - [M,], the equlibrium length of each bond.
- Outputs:
bond_ene (Tensor, float32) - [M,], the harmonic potential energy for each bond.
- Supported Platforms:
GPU
-
class
tinyms.primitives.BondForce(*args, **kwargs)[source]¶ Calculate the force exerted by the simple harmonic bond on the corresponding atoms. Assume the number of harmonic bonds is M and the number of atoms is N.
\[dr = (x_1-x_2, y_1-y_2, z_1-z_2)\]\[F = (F_x, F_y, F_z) = 2*k*(1 - r_0/|dr|)*dr\]- Parameters
atom_numbers (int32) – the number of atoms N.
bond_numbers (int32) – the number of harmonic bonds M.
- Inputs:
uint_crd_f (Tensor, uint32 ) - [N, 3], the unsigned int coordinate value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor (x, y, z), between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the first atom index of each bond.
atom_b (Tensor, int32) - [M,], the second atom index of each bond.
bond_k (Tensor, float32) - [M,], the force constant of each bond.
bond_r0 (Tensor, float32) - [M,], the equlibrium length of each bond.
- Outputs:
frc_f (float32 Tensor) - [N, 3], the force felt by each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.BondForceWithAtomEnergy(*args, **kwargs)[source]¶ Calculate bond force and harmonic potential energy together.
The calculation formula is the same as operator BondForce() and BondEnergy().
- Parameters
atom_numbers (int32) – the number of atoms N.
bond_numbers (int32) – the number of harmonic bonds M.
- Inputs:
uint_crd_f (Tensor, uint32 ) - [N, 3], the unsigned int coordinate value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor (x, y, z), between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the first atom index of each bond.
atom_b (Tensor, int32) - [M,], the second atom index of each bond.
bond_k (Tensor, float32) - [M,], the force constant of each bond.
bond_r0 (Tensor, float32) - [M,], the equlibrium length of each bond.
- Outputs:
frc_f (Tensor, float32) - [N, 3], same as operator BondForce().
atom_e (Tensor, float32) - [N,], same as atom_ene in operator BondAtomEnergy().
- Supported Platforms:
GPU
-
class
tinyms.primitives.BondForceWithAtomVirial(*args, **kwargs)[source]¶ Calculate bond force and the virial coefficient caused by simple harmonic bond for each atom together.
The calculation formula of the force part is the same as operator BondForce(). The Virial part is as follows:
\[dr = (x_1-x_2, y_1-y_2, z_1-z_2)\]\[virial = |dr|*(|dr| - r_0)*k\]- Parameters
atom_numbers (int32) – the number of atoms N.
bond_numbers (int32) – the number of harmonic bonds M.
- Inputs:
uint_crd_f (Tensor, uint32 ) - [N, 3], the unsigned int coordinate value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor (x, y, z), between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the first atom index of each bond.
atom_b (Tensor, int32) - [M,], the second atom index of each bond.
bond_k (Tensor, float32) - [M,], the force constant of each bond.
bond_r0 (Tensor, float32) - [M,], the equlibrium length of each bond.
- Outputs:
frc_f (Tensor, float32) - [N, 3], same as operator BondForce().
atom_v (Tensor, float32) - [N,], the accumulated virial coefficient for each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.BoundingBoxDecode(*args, **kwargs)[source]¶ Decodes bounding boxes locations.
- Parameters
means (tuple) – The means of deltas calculation. Default: (0.0, 0.0, 0.0, 0.0).
stds (tuple) – The standard deviations of deltas calculation. Default: (1.0, 1.0, 1.0, 1.0).
max_shape (tuple) – The max size limit for decoding box calculation.
wh_ratio_clip (float) – The limit of width and height ratio for decoding box calculation. Default: 0.016.
- Inputs:
anchor_box (Tensor) - Anchor boxes. The shape of anchor_box must be (n, 4).
deltas (Tensor) - Delta of boxes. Which has the same shape with anchor_box.
- Outputs:
Tensor, decoded boxes.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> anchor_box = Tensor([[4, 1, 2, 1], [2, 2, 2, 3]], mindspore.float32) >>> deltas = Tensor([[3, 1, 2, 2], [1, 2, 1, 4]], mindspore.float32) >>> boundingbox_decode = ops.BoundingBoxDecode(means=(0.0, 0.0, 0.0, 0.0), stds=(1.0, 1.0, 1.0, 1.0), ... max_shape=(768, 1280), wh_ratio_clip=0.016) >>> output = boundingbox_decode(anchor_box, deltas) >>> print(output) [[ 4.1953125 0. 0. 5.1953125] [ 2.140625 0. 3.859375 60.59375 ]]
-
class
tinyms.primitives.BoundingBoxEncode(*args, **kwargs)[source]¶ Encodes bounding boxes locations.
- Parameters
- Inputs:
anchor_box (Tensor) - Anchor boxes. The shape of anchor_box must be (n, 4).
groundtruth_box (Tensor) - Ground truth boxes. Which has the same shape with anchor_box.
- Outputs:
Tensor, encoded bounding boxes.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> anchor_box = Tensor([[2, 2, 2, 3], [2, 2, 2, 3]], mindspore.float32) >>> groundtruth_box = Tensor([[1, 2, 1, 4], [1, 2, 1, 4]], mindspore.float32) >>> boundingbox_encode = ops.BoundingBoxEncode(means=(0.0, 0.0, 0.0, 0.0), stds=(1.0, 1.0, 1.0, 1.0)) >>> output = boundingbox_encode(anchor_box, groundtruth_box) >>> print(output) [[ -1. 0.25 0. 0.40551758] [ -1. 0.25 0. 0.40551758]]
-
class
tinyms.primitives.Broadcast(*args, **kwargs)[source]¶ Broadcasts the tensor to the whole group.
Note
The tensors must have the same shape and format in all processes of the collection.
- Parameters
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape of the input, i.e., \((x_1, x_2, ..., x_R)\). The contents depend on the data of the root_rank device.
- Raises
TypeError – If root_rank is not a integer or group is not a string.
- Supported Platforms:
AscendGPU
Examples
>>> # This example should be run with multiple processes. >>> # Please refer to the tutorial > Distributed Training on mindspore.cn. >>> from mindspore import Tensor >>> from mindspore import context >>> from mindspore.communication import init >>> import mindspore.nn as nn >>> import mindspore.ops.operations as ops >>> import numpy as np >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.broadcast = ops.Broadcast(1) ... ... def construct(self, x): ... return self.broadcast((x,)) ... >>> input_ = Tensor(np.ones([2, 4]).astype(np.int32)) >>> net = Net() >>> output = net(input_) >>> print(output) (Tensor(shape[2,4], dtype=Int32, value= [[1, 1, 1, 1], [1, 1, 1, 1]]),)
-
class
tinyms.primitives.BroadcastTo(*args, **kwargs)[source]¶ Broadcasts input tensor to a given shape.
Input shape can be broadcast to target shape if for each dimension pair they are either equal or input is one or the target dimension is -1. In case of -1 in target shape, it will be replaced by the input shape’s value in that dimension.
When input shape is broadcast to target shape, it starts with the trailing dimensions.
- Parameters
shape (tuple) – The target shape to broadcast. Can be fully specified, or have -1 in one position where it will be substituted by the input tensor’s shape in that position, see example.
- Inputs:
input_x (Tensor) - The input tensor. The data type should be one of the following types: float16, float32, int32, int8, uint8.
- Outputs:
Tensor, with the given shape and the same data type as input_x.
- Raises
TypeError – If shape is not a tuple.
ValueError – Given a shape tuple, if it has several -1; or if the -1 is in an invalid position such as one that does not have a opposing dimension in an input tensor; or if the target and input shapes are incompatible.
- Supported Platforms:
AscendGPU
Examples
>>> shape = (2, 3) >>> input_x = Tensor(np.array([1, 2, 3]).astype(np.float32)) >>> broadcast_to = ops.BroadcastTo(shape) >>> output = broadcast_to(input_x) >>> print(output) [[1. 2. 3.] [1. 2. 3.]]
>>> shape = (2, -1) >>> input_x = Tensor(np.array([1, 2, 3]).astype(np.float32)) >>> broadcast_to = ops.BroadcastTo(shape) >>> output = broadcast_to(input_x) >>> print(output) [[1. 2. 3.] [1. 2. 3.]]
-
class
tinyms.primitives.CTCGreedyDecoder(*args, **kwargs)[source]¶ Performs greedy decoding on the logits given in inputs.
- Parameters
merge_repeated (bool) – If true, merge repeated classes in output. Default: True.
- Inputs:
inputs (Tensor) - The input Tensor must be a 3-D tensor whose shape is (max_time, batch_size, num_classes). num_classes must be num_labels + 1 classes, num_labels indicates the number of actual labels. Blank labels are reserved. Default blank label is num_classes - 1. Data type must be float32 or float64.
sequence_length (Tensor) - A tensor containing sequence lengths with the shape of (batch_size). The type must be int32. Each value in the tensor must be equal to or less than max_time.
- Outputs:
decoded_indices (Tensor) - A tensor with shape of (total_decoded_outputs, 2). Data type is int64.
decoded_values (Tensor) - A tensor with shape of (total_decoded_outputs), it stores the decoded classes. Data type is int64.
decoded_shape (Tensor) - The value of tensor is [batch_size, max_decoded_legth]. Data type is int64.
log_probability (Tensor) - A tensor with shape of (batch_size, 1), containing sequence log-probability, has the same type as inputs.
- Raises
TypeError – If merge_repeated is not a bool.
ValueError – If length of shape of inputs is not equal to 3.
ValueError – If length of shape of sequence_length is not equal to 1.
- Supported Platforms:
Ascend
Examples
>>> inputs = Tensor(np.random.random((2, 2, 3)), mindspore.float32) >>> sequence_length = Tensor(np.array([2, 2]), mindspore.int32) >>> ctc_greedy_decoder = ops.CTCGreedyDecoder() >>> out1, out2, out3, out4 = ctc_greedy_decoder(inputs, sequence_length) >>> print(out1, out2, out3, out4) [[0 0] [0 1] [1 0]] [0 1 0] [2 2] [[-0.7443749] [0.18251707]]
-
class
tinyms.primitives.CTCLoss(*args, **kwargs)[source]¶ Calculates the CTC (Connectionist Temporal Classification) loss and the gradient.
The CTC algorithm is proposed in Connectionist Temporal Classification: Labeling Unsegmented Sequence Data with Recurrent Neural Networks.
- Parameters
preprocess_collapse_repeated (bool) – If true, repeated labels will be collapsed prior to the CTC calculation. Default: False.
ctc_merge_repeated (bool) – If false, during CTC calculation, repeated non-blank labels will not be merged and these labels will be interpreted as individual ones. This is a simplfied version of CTC. Default: True.
ignore_longer_outputs_than_inputs (bool) – If true, sequences with longer outputs than inputs will be ignored. Default: False.
- Inputs:
inputs (Tensor) - The input Tensor must be a 3-D tensor whose shape is (max_time, batch_size, num_classes). num_classes must be num_labels + 1 classes, num_labels indicates the number of actual labels. Blank labels are reserved. Default blank label is num_classes - 1. Data type must be float16, float32 or float64.
labels_indices (Tensor) - The indices of labels. labels_indices[i, :] == [b, t] means labels_values[i] stores the id for (batch b, time t). The type must be int64 and rank must be 2.
labels_values (Tensor) - A 1-D input tensor. The values are associated with the given batch and time. The type must be int32. labels_values[i] must in the range of [0, num_classes).
sequence_length (Tensor) - A tensor containing sequence lengths with the shape of (batch_size). The type must be int32. Each value in the tensor must not be greater than max_time.
- Outputs:
loss (Tensor) - A tensor containing log-probabilities, the shape is (batch_size). The tensor has the same type with inputs.
gradient (Tensor) - The gradient of loss, has the same type and shape with inputs.
- Raises
TypeError – If preprocess_collapse_repeated, ctc_merge_repeated or ignore_longer_outputs_than_inputs is not a bool.
TypeError – If inputs, labels_indices, labels_values or sequence_length is not a Tensor.
TypeError – If dtype of inputs is not one of the following: float16, float32 or float64.
TypeError – If dtype of labels_indices is not int64.
TypeError – If dtype of labels_values or sequence_length is not int32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> np.random.seed(0) >>> inputs = Tensor(np.random.random((2, 2, 3)), mindspore.float32) >>> labels_indices = Tensor(np.array([[0, 0], [1, 0]]), mindspore.int64) >>> labels_values = Tensor(np.array([2, 2]), mindspore.int32) >>> sequence_length = Tensor(np.array([2, 2]), mindspore.int32) >>> ctc_loss = ops.CTCLoss() >>> loss, gradient = ctc_loss(inputs, labels_indices, labels_values, sequence_length) >>> print(loss) [ 0.7864997 0.720426 ] >>> print(gradient) [[[ 0.30898064 0.36491138 -0.673892 ] [ 0.33421117 0.2960548 -0.63026595 ]] [[ 0.23434742 0.36907154 0.11261538 ] [ 0.27316454 0.41090325 0.07584976 ]]]
-
class
tinyms.primitives.Cast(*args, **kwargs)[source]¶ Returns a tensor with the new specified data type.
- Inputs:
input_x (Union[Tensor, Number]) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The tensor to be cast.
type (dtype.Number) - The valid data type of the output tensor. Only constant value is allowed.
- Outputs:
Tensor, the shape of tensor is the same as input_x, \((x_1, x_2, ..., x_R)\).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_np = np.random.randn(2, 3, 4, 5).astype(np.float32) >>> input_x = Tensor(input_np) >>> type_dst = mindspore.int32 >>> cast = ops.Cast() >>> output = cast(input_x, type_dst) >>> print(output.dtype) Int32 >>> print(output.shape) (2, 3, 4, 5)
-
class
tinyms.primitives.Ceil(*args, **kwargs)[source]¶ Rounds a tensor up to the closest integer element-wise.
\[out_i = \lceil x_i \rceil = \lfloor x_i \rfloor + 1\]- Inputs:
input_x (Tensor) - The input tensor. It’s element data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If dtype of input_x is neither float16 nor float32.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([1.1, 2.5, -1.5]), mindspore.float32) >>> ceil_op = ops.Ceil() >>> output = ceil_op(input_x) >>> print(output) [ 2. 3. -1.]
-
class
tinyms.primitives.CheckBprop(*args, **kwargs)[source]¶ Checks whether the data type and the shape of corresponding elements from tuples x and y are the same.
- Inputs:
input_x (tuple[Tensor]) - The input_x contains the outputs of bprop to be checked.
input_y (tuple[Tensor]) - The input_y contains the inputs of bprop to check against.
- Outputs:
(tuple[Tensor]), the input_x, if data type and shape of corresponding elements from input_x and input_y are the same.
- Raises
TypeError – If input_x or input_y is not a Tensor.
Examples
>>> input_x = (Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32),) >>> input_y = (Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32),) >>> out = ops.CheckBprop()(input_x, input_y)
-
class
tinyms.primitives.CheckValid(*args, **kwargs)[source]¶ Checks bounding box.
Checks whether the bounding box cross data and data border are valid.
- Inputs:
bboxes (Tensor) - Bounding boxes tensor with shape (N, 4). Data type must be float16 or float32.
img_metas (Tensor) - Raw image size information with the format of (height, width, ratio). Data type must be float16 or float32.
- Outputs:
Tensor, with shape of (N,) and dtype of bool.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.check_valid = ops.CheckValid() ... def construct(self, x, y): ... valid_result = self.check_valid(x, y) ... return valid_result ... >>> bboxes = Tensor(np.linspace(0, 6, 12).reshape(3, 4), mindspore.float32) >>> img_metas = Tensor(np.array([2, 1, 3]), mindspore.float32) >>> net = Net() >>> output = net(bboxes, img_metas) >>> print(output) [ True False False]
-
class
tinyms.primitives.ComputeAccidentalHits(*args, **kwargs)[source]¶ Compute accidental hits of sampled classes which happen to match target classes.
When a target class matches the sample class, we call it “accidental hit”. The result of calculating accidental hit contains three parts (index, id, weight), where index represents the row number in true_classes, and id represents the position in sampled_candidates, the weight is -FLOAT_MAX.
- Parameters
num_true (int) – The number of target classes per training example.
- Inputs:
true_classes (Tensor) - The target classes. With data type of int32 or int64 and shape [batch_size, num_true].
sampled_candidates (Tensor) - The sampled_candidates output of CandidateSampler, with data type of int32 or int64 and shape [num_sampled].
- Outputs:
Tuple of 3 Tensors.
indices (Tensor) - A Tensor with shape (num_accidental_hits,), with the same type as true_classes.
ids (Tensor) - A Tensor with shape (num_accidental_hits,), with the same type as true_classes.
weights (Tensor) - A Tensor with shape (num_accidental_hits,), with the type float32.
- Raises
- Supported Platforms:
Ascend
Examples
>>> x = np.array([[1, 2], [0, 4], [3, 3]]) >>> y = np.array([0, 1, 2, 3, 4]) >>> sampler = ops.ComputeAccidentalHits(2) >>> output1, output2, output3 = sampler(Tensor(x), Tensor(y)) >>> print(output1, output2, output3) [0 0 1 1 2 2] [1 2 0 4 3 3] [-3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38 -3.4028235e+38]
-
class
tinyms.primitives.Concat(*args, **kwargs)[source]¶ Connect tensor in the specified axis.
Connect input tensors along with the given axis.
The input data is a tuple of tensors. These tensors have the same rank R. Set the given axis as m, and \(0 \le m < R\). Set the number of input tensors as N. For the \(i\)-th tensor \(t_i\), it has the shape of \((x_1, x_2, ..., x_{mi}, ..., x_R)\). \(x_{mi}\) is the \(m\)-th dimension of the \(i\)-th tensor. Then, the shape of the output tensor is
\[(x_1, x_2, ..., \sum_{i=1}^Nx_{mi}, ..., x_R)\]- Parameters
axis (int) – The specified axis. Default: 0.
- Inputs:
input_x (tuple, list) - A tuple or a list of input tensors.
- Outputs:
Tensor, the shape is \((x_1, x_2, ..., \sum_{i=1}^Nx_{mi}, ..., x_R)\).
- Raises
TypeError – If axis is not an int.
- Supported Platforms:
AscendGPUCPU
Examples
>>> data1 = Tensor(np.array([[0, 1], [2, 1]]).astype(np.float32)) >>> data2 = Tensor(np.array([[0, 1], [2, 1]]).astype(np.float32)) >>> op = ops.Concat() >>> output = op((data1, data2)) >>> print(output) [[0. 1.] [2. 1.] [0. 1.] [2. 1.]]
-
class
tinyms.primitives.ConfusionMatrix(*args, **kwargs)[source]¶ Calculates the confusion matrix from labels and predictions.
- Parameters
- Inputs:
labels (Tensor) - real labels, tensor of 1-D. the dtype must be non-negative Integer.
predictions (Tensor) - the labels from prediction, tensor of 1-D. the shape same as labels and the dtype must be non-negative Integer.
weights (Tensor) - tensor of 1-D. the shape same as predictions.
- Outputs:
Tensor, the confusion matrix, with shape (num_classes, num_classes).
- Raises
Examples
>>> confusion_matrix = ops.ConfusionMatrix(4) >>> labels = Tensor([0, 1, 1, 3], mindspore.int32) >>> predictions = Tensor([1, 2, 1, 3], mindspore.int32) >>> output = confusion_matrix(labels, predictions) >>> print(output) [[0 1 0 0] [0 1 1 0] [0 0 0 0] [0 0 0 1]]
-
class
tinyms.primitives.Conv2D(*args, **kwargs)[source]¶ 2D convolution layer.
Applies a 2D convolution over an input tensor which is typically of shape \((N, C_{in}, H_{in}, W_{in})\), where \(N\) is batch size and \(C_{in}\) is channel number. For each batch of shape \((C_{in}, H_{in}, W_{in})\), the formula is defined as:
\[out_j = \sum_{i=0}^{C_{in} - 1} ccor(W_{ij}, X_i) + b_j,\]where \(ccor\) is the cross correlation operator, \(C_{in}\) is the input channel number, \(j\) ranges from \(0\) to \(C_{out} - 1\), \(W_{ij}\) corresponds to the \(i\)-th channel of the \(j\)-th filter and \(out_{j}\) corresponds to the \(j\)-th channel of the output. \(W_{ij}\) is a slice of kernel and it has shape \((\text{ks_h}, \text{ks_w})\), where \(\text{ks_h}\) and \(\text{ks_w}\) are the height and width of the convolution kernel. The full kernel has shape \((C_{out}, C_{in} // \text{group}, \text{ks_h}, \text{ks_w})\), where group is the group number to split the input in the channel dimension.
If the ‘pad_mode’ is set to be “valid”, the output height and width will be \(\left \lfloor{1 + \frac{H_{in} + 2 \times \text{padding} - \text{ks_h} - (\text{ks_h} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) and \(\left \lfloor{1 + \frac{W_{in} + 2 \times \text{padding} - \text{ks_w} - (\text{ks_w} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) respectively.
The first introduction can be found in paper Gradient Based Learning Applied to Document Recognition. More detailed introduction can be found here: http://cs231n.github.io/convolutional-networks/.
- Parameters
out_channel (int) – The dimension of the output.
kernel_size (Union[int, tuple[int]]) – The kernel size of the 2D convolution.
mode (int) – Modes for different convolutions. 0 Math convolutiuon, 1 cross-correlation convolution , 2 deconvolution, 3 depthwise convolution. Default: 1.
pad_mode (str) – Modes to fill padding. It could be “valid”, “same”, or “pad”. Default: “valid”.
pad (Union(int, tuple[int])) – The pad value to be filled. Default: 0. If pad is an integer, the paddings of top, bottom, left and right are the same, equal to pad. If pad is a tuple of four integers, the padding of top, bottom, left and right equal to pad[0], pad[1], pad[2], and pad[3] correspondingly.
stride (Union(int, tuple[int])) – The stride to be applied to the convolution filter. Default: 1.
dilation (Union(int, tuple[int])) – Specifies the space to use between kernel elements. Default: 1.
group (int) – Splits input into groups. Default: 1.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: “NCHW”.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
weight (Tensor) - Set size of kernel is \((K_1, K_2)\), then the shape is \((C_{out}, C_{in}, K_1, K_2)\).
- Outputs:
Tensor, the value that applied 2D convolution. The shape is \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If kernel_size, stride, pad or dilation is neither an int nor a tuple.
TypeError – If out_channel or group is not an int.
ValueError – If kernel_size, stride or dilation is less than 1.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
ValueError – If pad is a tuple whose length is not equal to 4.
ValueError – If pad_mode it not equal to ‘pad’ and pad is not equal to (0, 0, 0, 0).
ValueError – If data_format is neither ‘NCHW’ not ‘NHWC’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input = Tensor(np.ones([10, 32, 32, 32]), mindspore.float32) >>> weight = Tensor(np.ones([32, 32, 3, 3]), mindspore.float32) >>> conv2d = ops.Conv2D(out_channel=32, kernel_size=3) >>> output = conv2d(input, weight) >>> print(output.shape) (10, 32, 30, 30)
-
class
tinyms.primitives.Conv2DBackpropInput(*args, **kwargs)[source]¶ Computes the gradients of convolution with respect to the input.
- Parameters
out_channel (int) – The dimensionality of the output space.
kernel_size (Union[int, tuple[int]]) – The size of the convolution window.
pad_mode (str) – Modes to fill padding. It could be “valid”, “same”, or “pad”. Default: “valid”.
pad (Union[int, tuple[int]]) – The pad value to be filled. Default: 0. If pad is an integer, the paddings of top, bottom, left and right are the same, equal to pad. If pad is a tuple of four integers, the padding of top, bottom, left and right equal to pad[0], pad[1], pad[2], and pad[3] correspondingly.
mode (int) – Modes for different convolutions. 0 Math convolutiuon, 1 cross-correlation convolution , 2 deconvolution, 3 depthwise convolution. Default: 1.
stride (Union[int. tuple[int]]) – The stride to be applied to the convolution filter. Default: 1.
dilation (Union[int. tuple[int]]) – Specifies the dilation rate to be used for the dilated convolution. Default: 1.
group (int) – Splits input into groups. Default: 1.
data_format (str) –
- Inputs:
dout (Tensor) - the gradients w.r.t the output of the convolution. The shape conforms to the default data_format \((N, C_{out}, H_{out}, W_{out})\).
weight (Tensor) - Set size of kernel is \((K_1, K_2)\), then the shape is \((C_{out}, C_{in}, K_1, K_2)\).
input_size (Tensor) - A tuple describes the shape of the input which conforms to the format \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, the gradients w.r.t the input of convolution. It has the same shape as the input.
- Raises
TypeError – If kernel_size, stride, pad or dilation is neither an int nor a tuple.
TypeError – If out_channel or group is not an int.
ValueError – If kernel_size, stride or dilation is less than 1.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
ValueError – If padding is a tuple whose length is not equal to 4.
ValueError – If pad_mode it not equal to ‘pad’ and pad is not equal to (0, 0, 0, 0).
ValueError – If data_format is neither ‘NCHW’ not ‘NHWC’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> dout = Tensor(np.ones([10, 32, 30, 30]), mindspore.float32) >>> weight = Tensor(np.ones([32, 32, 3, 3]), mindspore.float32) >>> x = Tensor(np.ones([10, 32, 32, 32])) >>> conv2d_backprop_input = ops.Conv2DBackpropInput(out_channel=32, kernel_size=3) >>> output = conv2d_backprop_input(dout, weight, F.shape(x)) >>> print(output.shape) (10, 32, 32, 32)
-
class
tinyms.primitives.Conv3D(*args, **kwargs)[source]¶ 3D convolution layer.
Applies a 3D convolution over an input tensor which is typically of shape \((N, C_{in}, D_{in}, H_{in}, W_{in})\) and output shape \((N, C_{out}, D_{out}, H_{out}, W_{out})\). where \(N\) is batch size. \(C\) is channel number. the formula is defined as:
\[\operatorname{out}\left(N_{i}, C_{\text {out}_j}\right)=\operatorname{bias}\left(C_{\text {out}_j}\right)+ \sum_{k=0}^{C_{in}-1} ccor(\text {weight}\left(C_{\text {out}_j}, k\right), \operatorname{input}\left(N_{i}, k\right))\]where \(ccor\) is the cross-correlation operator.
If the ‘pad_mode’ is set to be “valid”, the output height and width will be \(\left \lfloor{1 + \frac{D_{in} + 2 \times \text{padding} - \text{ks_d} - (\text{ks_d} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) and \(\left \lfloor{1 + \frac{H_{in} + 2 \times \text{padding} - \text{ks_h} - (\text{ks_h} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) and \(\left \lfloor{1 + \frac{W_{in} + 2 \times \text{padding} - \text{ks_w} - (\text{ks_w} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) respectively.
- Parameters
out_channels (int) – The number of output channel \(C_{out}\).
kernel_size (Union[int, tuple[int]]) – The data type is int or a tuple of 3 integers. Specifies the depth, height and width of the 3D convolution window. Single int means the value is for the depth, height and the width of the kernel. A tuple of 3 ints means the first value is for the depth, height and the other is for the width of the kernel.
mode (int) – Modes for different convolutions. Not currently used.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the depth, height and width of movement are both strides, or a tuple of three int numbers that represent depth, height and width of movement respectively. Default: 1.
pad_mode (str) –
Specifies padding mode. The optional values are “same”, “valid”, “pad”. Default: “valid”.
same: Adopts the way of completion. The depth, height and width of the output will be the same as the input. The total number of padding will be calculated in depth, horizontal and vertical directions and evenly distributed to head and tail, top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the tail, bottom and the right side. If this mode is set, pad must be 0.
valid: Adopts the way of discarding. The possible largest depth, height and width of output will be returned without padding. Extra pixels will be discarded. If this mode is set, pad must be 0.
pad: Implicit paddings on both sides of the input in depth, height, width. The number of pad will be padded to the input Tensor borders. pad must be greater than or equal to 0.
pad (Union(int, tuple[int])) – The pad value to be filled. Default: 0. If pad is an integer, the paddings of head, tail, top, bottom, left and right are the same, equal to pad. If pad is a tuple of six integers, the padding of head, tail, top, bottom, left and right equal to pad[0], pad[1], pad[2], pad[3], pad[4] and pad[5] correspondingly.
dilation (Union[int, tuple[int]]) – The data type is int or a tuple of 3 integers : math:(dilation_d, dilation_h, dilation_w). Currently, dilation on depth only supports the case of 1. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater or equal to 1 and bounded by the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_ channels and out_channels must be divisible by the number of groups. Default: 1. Only 1 is currently supported.
data_format (str) – The optional value for data format. Currently only support “NCDHW”.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, D_{in}, H_{in}, W_{in})\). Currently input data type only support float16 and float32.
weight (Tensor) - Set size of kernel is \((k_d, K_h, K_w)\), then the shape is \((C_{out}, C_{in}//groups, k_d, K_h, K_w)\). Currently weight data type only support float16 and float32.
bias (Tensor) - Tensor of shape \(C_{in}\). Currently, only support none.
- Outputs:
Tensor, the value that applied 3D convolution. The shape is \((N, C_{out}, D_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If out_channel or group is not an int.
TypeError – If kernel_size, stride, pad or dilation is neither an int nor a tuple of six.
ValueError – If out_channel, kernel_size, stride or dilation is less than 1.
ValueError – If pad is less than 0.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
ValueError – If pad is a tuple whose length is not equal to 6.
ValueError – If pad_mode is not equal to ‘pad’ and pad is not equal to (0, 0, 0, 0, 0, 0).
ValueError – If data_format is not ‘NCDHW’.
- Supported Platforms:
Ascend
Examples
>>> input = Tensor(np.ones([16, 3, 10, 32, 32]), mindspore.float16) >>> weight = Tensor(np.ones([32, 3, 4, 3, 3]), mindspore.float16) >>> conv3d = P.Conv3D(out_channel=32, kernel_size=(4, 3, 3)) >>> output = conv3d(input, weight) >>> print(output.shape) (16, 32, 7, 30, 30)
-
class
tinyms.primitives.Conv3DTranspose(*args, **kwargs)[source]¶ Compute a 3D transposed convolution, which is also known as a deconvolution (although it is not an actual deconvolution).
Input is typically of shape \((N, C, D, H, W)\), where \(N\) is batch size and \(C\) is channel number.
If the ‘pad_mode’ is set to be “pad”, the height and width of output are defined as:
\[ \begin{align}\begin{aligned}D_{out} = (D_{in} - 1) \times \text{stride_d} - 2 \times \text{padding_d} + \text{dilation_d} \times (\text{kernel_size_d} - 1) + \text{output_padding_d} + 1\\H_{out} = (H_{in} - 1) \times \text{stride_h} - 2 \times \text{padding_h} + \text{dilation_h} \times (\text{kernel_size_h} - 1) + \text{output_padding_h} + 1\\W_{out} = (W_{in} - 1) \times \text{stride_w} - 2 \times \text{padding_w} + \text{dilation_w} \times (\text{kernel_size_w} - 1) + \text{output_padding_w} + 1\end{aligned}\end{align} \]- Parameters
in_channel (int) – The channel of the input x.
out_channel (int) – The channel of the weight x.
kernel_size (Union[int, tuple[int]]) – The kernel size of the 3D convolution.
mode (int) – Modes for different convolutions. Default is 1. Not currently used.
pad_mode (str) –
Specifies padding mode. The optional values are “same”, “valid”, “pad”. Default: “valid”.
same: Adopts the way of completion. The depth, height and width of the output will be the same as the input. The total number of padding will be calculated in depth, horizontal and vertical directions and evenly distributed to head and tail, top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the tail, bottom and the right side. If this mode is set, pad and output_padding must be 0.
valid: Adopts the way of discarding. The possible largest depth, height and width of output will be returned without padding. Extra pixels will be discarded. If this mode is set, pad and output_padding must be 0.
pad: Implicit paddings on both sides of the input in depth, height, width. The number of pad will be padded to the input Tensor borders. pad must be greater than or equal to 0.
pad (Union(int, tuple[int])) – The pad value to be filled. Default: 0. If pad is an integer, the paddings of head, tail, top, bottom, left and right are the same, equal to pad. If pad is a tuple of six integers, the padding of head, tail, top, bottom, left and right equal to pad[0], pad[1], pad[2], pad[3], pad[4] and pad[5] correspondingly.
stride (Union(int, tuple[int])) – The stride to be applied to the convolution filter. Default: 1.
dilation (Union(int, tuple[int])) – Specifies the space to use between kernel elements. Default: 1.
group (int) – Splits input into groups. Default: 1. Only 1 is currently supported.
output_padding (Union(int, tuple[int])) – Add extra size to each dimension of the output. Default: 0.
data_format (str) – The optional value for data format. Currently only support ‘NCDHW’.
- Inputs:
dout (Tensor) - the gradients w.r.t the output of the convolution. The shape conforms to the default data_format \((N, C_{in}, D_{out}, H_{out}, W_{out})\). Currently dout data type only support float16 and float32.
weight (Tensor) - Set size of kernel is \((k_d, K_h, K_w)\), then the shape is \((C_{in}, C_{out}//groups, k_d, K_h, K_w)\). Currently weight data type only support float16 and float32.
bias (Tensor) - Tensor of shape \(C_{out}\). Currently, only support none.
- Outputs:
Tensor, the gradients w.r.t the input of convolution 3D. It has the same shape as the input.
- Supported Platforms:
Ascend
- Raises
TypeError – If in_channel, out_channel or group is not an int.
TypeError – If kernel_size, stride, pad , dilation or output_padding is neither an int not a tuple.
ValueError – If in_channel, out_channel, kernel_size, stride or dilation is less than 1.
ValueError – If pad is less than 0.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
ValueError – If pad is a tuple whose length is not equal to 6.
ValueError – If pad_mode is not equal to ‘pad’ and pad is not equal to (0, 0, 0, 0, 0, 0).
ValueError – If data_format is not ‘NCDHW’.
TypeError – If dout and weight data type not float16.
ValueError – If bias not none. The rank of dout and weight is not 5.
Examples
>>> input_x = Tensor(np.ones([32, 16, 10, 32, 32]), mindspore.float16) >>> weight = Tensor(np.ones([16, 3, 4, 6, 2]), mindspore.float16) >>> conv3d_transpose = P.Conv3DTranspose(in_channel=16, out_channel=3, kernel_size=(4, 6, 2)) >>> output = conv3d_transpose(input_x, weight) >>> print(output.shape) (32, 3, 13, 37, 33)
-
class
tinyms.primitives.Cos(*args, **kwargs)[source]¶ Computes cosine of input element-wise.
\[out_i = cos(x_i)\]- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> cos = ops.Cos() >>> input_x = Tensor(np.array([0.24, 0.83, 0.31, 0.09]), mindspore.float32) >>> output = cos(input_x) >>> print(output) [0.971338 0.67487574 0.95233357 0.9959527 ]
-
class
tinyms.primitives.Cosh(*args, **kwargs)[source]¶ Computes hyperbolic cosine of input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendCPU
Examples
>>> cosh = ops.Cosh() >>> input_x = Tensor(np.array([0.24, 0.83, 0.31, 0.09]), mindspore.float32) >>> output = cosh(input_x) >>> print(output) [1.0289385 1.364684 1.048436 1.0040528]
-
class
tinyms.primitives.CrdToUintCrd(*args, **kwargs)[source]¶ Convert FP32 coordinate to Uint32 coordinate.
- Parameters
atom_numbers (int32) – the number of atoms N.
- Inputs:
crd_to_uint_crd_cof (Tensor, float32) - [3,], the .
crd (Tensor, float32) - [N, 3], the coordinate of each atom.
- Outputs:
output (uint32)
- Supported Platforms:
GPU
-
class
tinyms.primitives.CropAndResize(*args, **kwargs)[source]¶ Extracts crops from the input image tensor and resizes them.
Note
In case that the output shape depends on crop_size, the crop_size must be constant.
- Parameters
method (str) – An optional string that specifies the sampling method for resizing. It can be “bilinear”, “nearest” or “bilinear_v2”. The option “bilinear” stands for standard bilinear interpolation algorithm, while “bilinear_v2” may result in better result in some cases. Default: “bilinear”
extrapolation_value (float) – An optional float value used extrapolation, if applicable. Default: 0.
- Inputs:
x (Tensor) - The input image must be a 4-D tensor of shape [batch, image_height, image_width, depth]. Types allowed: int8, int16, int32, int64, float16, float32, float64, uint8, uint16.
boxes (Tensor) - A 2-D tensor of shape [num_boxes, 4]. The i-th row of the tensor specifies the coordinates of a box in the box_ind[i] image and is specified in normalized coordinates [y1, x1, y2, x2]. A normalized coordinate value of y is mapped to the image coordinate at y * (image_height - 1), so as the [0, 1] interval of normalized image height is mapped to [0, image_height - 1] in image height coordinates. We do allow y1 > y2, in which case the sampled crop is an up-down flipped version of the original image. The width dimension is treated similarly. Normalized coordinates outside the [0, 1] range are allowed, in which case we use extrapolation_value to extrapolate the input image values. Types allowed: float32.
box_index (Tensor) - A 1-D tensor of shape [num_boxes] with int32 values in [0, batch). The value of box_ind[i] specifies the image that the i-th box refers to. Types allowed: int32.
crop_size (Tuple[int]) - A tuple of two int32 elements: (crop_height, crop_width). Only constant value is allowed. All cropped image patches are resized to this size. The aspect ratio of the image content is not preserved. Both crop_height and crop_width need to be positive.
- Outputs:
A 4-D tensor of shape [num_boxes, crop_height, crop_width, depth] with type: float32.
- Raises
TypeError – If method is not a str.
TypeError – If extrapolation_value is not a float.
ValueError – If method is not one of ‘bilinear’, ‘nearest’, ‘bilinear_v2’.
- Supported Platforms:
Ascend
Examples
>>> class CropAndResizeNet(nn.Cell): ... def __init__(self, crop_size): ... super(CropAndResizeNet, self).__init__() ... self.crop_and_resize = ops.CropAndResize() ... self.crop_size = crop_size ... ... def construct(self, x, boxes, box_index): ... return self.crop_and_resize(x, boxes, box_index, self.crop_size) ... >>> BATCH_SIZE = 1 >>> NUM_BOXES = 5 >>> IMAGE_HEIGHT = 256 >>> IMAGE_WIDTH = 256 >>> CHANNELS = 3 >>> image = np.random.normal(size=[BATCH_SIZE, IMAGE_HEIGHT, IMAGE_WIDTH, CHANNELS]).astype(np.float32) >>> boxes = np.random.uniform(size=[NUM_BOXES, 4]).astype(np.float32) >>> box_index = np.random.uniform(size=[NUM_BOXES], low=0, high=BATCH_SIZE).astype(np.int32) >>> crop_size = (24, 24) >>> crop_and_resize = CropAndResizeNet(crop_size=crop_size) >>> output = crop_and_resize(Tensor(image), Tensor(boxes), Tensor(box_index)) >>> print(output.shape) (5, 24, 24, 3)
-
class
tinyms.primitives.CumProd(*args, **kwargs)[source]¶ Computes the cumulative product of the tensor x along axis.
- Parameters
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (int) - The dimensions to compute the cumulative product. Only constant value is allowed.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Raises
TypeError – If exclusive or reverse is not a bool.
ValueError – If axis is None.
- Supported Platforms:
Ascend
Examples
>>> a, b, c, = 1, 2, 3 >>> input_x = Tensor(np.array([a, b, c]).astype(np.float32)) >>> op0 = ops.CumProd() >>> output0 = op0(input_x, 0) # output=[a, a * b, a * b * c] >>> op1 = ops.CumProd(exclusive=True) >>> output1 = op1(input_x, 0) # output=[1, a, a * b] >>> op2 = ops.CumProd(reverse=True) >>> output2 = op2(input_x, 0) # output=[a * b * c, b * c, c] >>> op3 = ops.CumProd(exclusive=True, reverse=True) >>> output3 = op3(input_x, 0) # output=[b * c, c, 1] >>> print(output0) [1. 2. 6.] >>> print(output1) [1. 1. 2.] >>> print(output2) [6. 6. 3.] >>> print(output3) [6. 3. 1.]
-
class
tinyms.primitives.CumSum(*args, **kwargs)[source]¶ Computes the cumulative sum of input tensor along axis.
\[y_i = x_1 + x_2 + x_3 + ... + x_i\]- Parameters
- Inputs:
input (Tensor) - The input tensor to accumulate.
axis (int) - The axis to accumulate the tensor’s value. Only constant value is allowed. Must be in the range [-rank(input), rank(input)).
- Outputs:
Tensor, the shape of the output tensor is consistent with the input tensor’s.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input = Tensor(np.array([[3, 4, 6, 10], [1, 6, 7, 9], [4, 3, 8, 7], [1, 3, 7, 9]]).astype(np.float32)) >>> cumsum = ops.CumSum() >>> output = cumsum(input, 1) >>> print(output) [[ 3. 7. 13. 23.] [ 1. 7. 14. 23.] [ 4. 7. 15. 22.] [ 1. 4. 11. 20.]]
-
class
tinyms.primitives.DType(*args, **kwargs)[source]¶ Returns the data type of the input tensor as mindspore.dtype.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
mindspore.dtype, the data type of a tensor.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> output = ops.DType()(input_tensor) >>> print(output) Float32
-
class
tinyms.primitives.DataFormatDimMap(*args, **kwargs)[source]¶ Returns the dimension index in the destination data format given in the source data format.
- Parameters
src_format (string) – An optional value for source data format. Default: ‘NHWC’.
dst_format (string) – An optional value for destination data format. Default: ‘NCHW’.
- Inputs:
input_x (Tensor) - A Tensor with each element as a dimension index in source data format. The suggested values is in the range [-4, 4). It’s type is int32.
- Outputs:
Tensor, has the same type as the input_x.
- Raises
- Supported Platforms:
Ascend
Examples
>>> x = Tensor([0, 1, 2, 3], mindspore.int32) >>> dfdm = ops.DataFormatDimMap() >>> output = dfdm(x) >>> print(output) [0 3 1 2]
-
class
tinyms.primitives.Depend(*args, **kwargs)[source]¶ Depend is used for processing dependency operations.
In most scenarios, if operators have IO side effects or memory side effects, they will be executed according to the user’s semantics. In some scenarios, if the two operators A and B have no order dependency, and A must be executed before B, we recommend using Depend to specify their execution order. The usage method is as follows:
a = A(x) ---> a = A(x) b = B(y) ---> y = Depend(y, a) ---> b = B(y)
- Inputs:
value (Tensor) - the real value to return for depend operator.
expr (Expression) - the expression to execute with no outputs.
- Outputs:
Tensor, the value passed by last operator.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import numpy as np >>> import mindspore >>> import mindspore.nn as nn >>> import mindspore.ops.operations as P >>> from mindspore import Tensor >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.softmax = P.Softmax() ... self.depend = P.Depend() ... ... def construct(self, x, y): ... mul = x * y ... y = self.depend(y, mul) ... ret = self.softmax(y) ... return ret ... >>> x = Tensor(np.ones([4, 5]), dtype=mindspore.float32) >>> y = Tensor(np.ones([4, 5]), dtype=mindspore.float32) >>> net = Net() >>> output = net(x, y) >>> print(output) [[0.2 0.2 0.2 0.2 0.2] [0.2 0.2 0.2 0.2 0.2] [0.2 0.2 0.2 0.2 0.2] [0.2 0.2 0.2 0.2 0.2]]
-
class
tinyms.primitives.DepthToSpace(*args, **kwargs)[source]¶ Rearranges blocks of depth data into spatial dimensions.
This is the reverse operation of SpaceToDepth.
The depth of output tensor is \(input\_depth / (block\_size * block\_size)\).
The output tensor’s height dimension is \(height * block\_size\).
The output tensor’s weight dimension is \(weight * block\_size\).
The input tensor’s depth must be divisible by block_size * block_size. The data format is “NCHW”.
- Parameters
block_size (int) – The block size used to divide depth data. It must be >= 2.
- Inputs:
x (Tensor) - The target tensor. It must be a 4-D tensor with shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{in} / \text{block_size}, H_{in} * \text{block_size}, W_{in} * \text{block_size})\).
- Raises
TypeError – If block_size is not an int.
ValueError – If block_size is less than 2.
ValueError – If length of shape of x is not equal to 4.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(1, 12, 1, 1), mindspore.float32) >>> block_size = 2 >>> depth_to_space = ops.DepthToSpace(block_size) >>> output = depth_to_space(x) >>> print(output.shape) (1, 3, 2, 2)
-
class
tinyms.primitives.DepthwiseConv2dNative(*args, **kwargs)[source]¶ Returns the depth-wise convolution value for the input.
Applies depthwise conv2d for the input, which will generate more channels with channel_multiplier. Given an input tensor of shape \((N, C_{in}, H_{in}, W_{in})\) where \(N\) is the batch size and a filter tensor with kernel size \((ks_{h}, ks_{w})\), containing \(C_{in} * \text{channel_multiplier}\) convolutional filters of depth 1; it applies different filters to each input channel (channel_multiplier channels for each input channel has the default value 1), then concatenates the results together. The output has \(\text{in_channels} * \text{channel_multiplier}\) channels.
- Parameters
channel_multiplier (int) – The multiplier for the original output convolution. Its value must be greater than 0.
kernel_size (Union[int, tuple[int]]) – The size of the convolution kernel.
mode (int) – Modes for different convolutions. 0 Math convolution, 1 cross-correlation convolution , 2 deconvolution, 3 depthwise convolution. Default: 3.
pad_mode (str) – Modes to fill padding. It could be “valid”, “same”, or “pad”. Default: “valid”.
pad (Union[int, tuple[int]]) – The pad value to be filled. If pad is an integer, the paddings of top, bottom, left and right are the same, equal to pad. If pad is a tuple of four integers, the padding of top, bottom, left and right equal to pad[0], pad[1], pad[2], and pad[3] correspondingly. Default: 0.
stride (Union[int, tuple[int]]) – The stride to be applied to the convolution filter. Default: 1.
dilation (Union[int, tuple[int]]) – Specifies the dilation rate to be used for the dilated convolution. Default: 1.
group (int) – Splits input into groups. Default: 1.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
weight (Tensor) - Set the size of kernel as \((K_1, K_2)\), then the shape is \((K, C_{in}, K_1, K_2)\), K must be 1.
- Outputs:
Tensor of shape \((N, C_{in} * \text{channel_multiplier}, H_{out}, W_{out})\).
- Raises
TypeError – If kernel_size, stride, pad or dilation is neither an int nor a tuple.
TypeError – If channel_multiplier or group is not an int.
ValueError – If stride or dilation is less than 1.
ValueError – If pad_mode is not one of the following:’same’, ‘valid’ or ‘pad’.
ValueError – If pad_mode it not equal to ‘pad’ and pad is not equal to (0, 0, 0, 0).
- Supported Platforms:
Ascend
Examples
>>> input = Tensor(np.ones([10, 32, 32, 32]), mindspore.float32) >>> weight = Tensor(np.ones([1, 32, 3, 3]), mindspore.float32) >>> depthwise_conv2d = ops.DepthwiseConv2dNative(channel_multiplier=3, kernel_size=(3, 3)) >>> output = depthwise_conv2d(input, weight) >>> print(output.shape) (10, 96, 30, 30)
-
class
tinyms.primitives.Diag(*args, **kwargs)[source]¶ Constructs a diagonal tensor with a given diagonal values.
Assume input_x has dimensions \([D_1,... D_k]\), the output is a tensor of rank 2k with dimensions \([D_1,..., D_k, D_1,..., D_k]\) where: \(output[i_1,..., i_k, i_1,..., i_k] = input_x[i_1,..., i_k]\) and 0 everywhere else.
- Inputs:
input_x (Tensor) - The input tensor. The input shape must be less than 5d.
- Outputs:
Tensor, has the same dtype as the input_x.
Examples
>>> input_x = Tensor([1, 2, 3, 4]) >>> diag = ops.Diag() >>> output = diag(input_x) >>> print(output) [[1, 0, 0, 0], [0, 2, 0, 0], [0, 0, 3, 0], [0, 0, 0, 4]]
-
class
tinyms.primitives.DiagPart(*args, **kwargs)[source]¶ Extracts the diagonal part from given tensor.
Assume input has dimensions \([D_1,..., D_k, D_1,..., D_k]\), the output is a tensor of rank k with dimensions \([D_1,..., D_k]\) where: \(output[i_1,..., i_k] = input[i_1,..., i_k, i_1,..., i_k]\).
- Inputs:
input_x (Tensor) - tensor of rank k where k is even and not zero.
- Outputs:
Tensor, the extracted diagonal has the same dtype as the input_x.
- Examples
>>> input_x = Tensor([[1, 0, 0, 0], ... [0, 2, 0, 0], ... [0, 0, 3, 0], ... [0, 0, 0, 4]]) >>> diag_part = ops.DiagPart() >>> output = diag_part(input_x) >>> print(output) [1 2 3 4]
-
class
tinyms.primitives.Dihedral14CFAtomEnergy(*args, **kwargs)[source]¶ Add the potential energy caused by Coulumb energy correction for each necessary dihedral 1,4 terms to the total potential energy of each atom.
The calculation formula is the same as operator Dihedral14CFEnergy().
- Parameters
nb14_numbers (int32) – the number of necessary dihedral 1,4 terms M.
atom_numbers (int32) – the number of atoms N.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJ_type (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge of each atom.
boxlength_f (Tensor, float32) - [3,], the length of molecular simulation box in 3 dimensions.
a_14 (Tensor, int32) - [M,], the first atom index of each dihedral 1,4 term.
b_14 (Tensor, int32) - [M,], the second atom index of each dihedral 1,4 term.
cf_scale_factor (Tensor, float) - [M,], the scale factor for the Coulomb part of force correction for each dihedral 1,4 terms.
- Outputs:
ene (Tensor, float32) - [N,], the accumulated potential energy of each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.Dihedral14CFEnergy(*args, **kwargs)[source]¶ Calculate the Coulumb part of 1,4 dihedral energy correction for each necessary dihedral terms on the corresponding atoms.
\[dr = (x_a-x_b, y_a-y_b, z_a-z_b)\]\[E = k*q_a*q_b/|dr|\]- Parameters
nb14_numbers (int32) – the number of necessary dihedral 1,4 terms M.
atom_numbers (int32) – the number of atoms N.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJ_type (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge of each atom.
boxlength_f (Tensor, float32) - [3,], the length of molecular simulation box in 3 dimensions.
a_14 (Tensor, int32) - [M,], the first atom index of each dihedral 1,4 term.
b_14 (Tensor, int32) - [M,], the second atom index of each dihedral 1,4 term.
cf_scale_factor (Tensor, float) - [M,], the scale factor for the Coulomb part of force correction for each dihedral 1,4 terms.
- Outputs:
ene (Tensor, float32) - [M,], the accumulated potential energy of each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.Dihedral14LJAtomEnergy(*args, **kwargs)[source]¶ Add the potenrial energy caused by Lennard-Jones energy correction for each necessary dihedral 1,4 terms to the total potential energy of each atom.
The calculation formula is the same as operator Dihedral14LJEnergy().
- Parameters
nb14_numbers (int32) – the number of necessary dihedral 1,4 terms M.
atom_numbers (int32) – the number of atoms N.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJ_type (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge of each atom.
boxlength_f (Tensor, float32) - [3,], the length of molecular simulation box in 3 dimensions.
a_14 (Tensor, int32) - [M,], the first atom index of each dihedral 1,4 term.
b_14 (Tensor, int32) - [M,], the second atom index of each dihedral 1,4 term.
lj_scale_factor (Tensor, float32) - [M,], the scale factor for the Lennard-Jones part of force correction of each dihedral 1,4 term.
LJ_type_A (Tensor, float32) - [Q,], the A parameter in Lennard-Jones scheme of each atom pair type. Q is the number of atom pair.
LJ_type_B (Tensor, float32) - [Q,], the B parameter in Lennard-Jones shceme of each atom pair type. Q is the number of atom pair.
- Outputs:
ene (Tensor, float32) - [N,], the accumulated potential energy of each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.Dihedral14LJCFForceWithAtomEnergy(*args, **kwargs)[source]¶ Calculate the Lennard-Jones and Coulumb energy correction and force correction for each necessary dihedral 1,4 terms together and add them to the total force and potential energy for each atom.
The calculation formula of force correction is the same as operator Dihedral14LJForceWithDirectCF(), and the energy correction part is the same as operator Dihedral14LJEnergy() and Dihedral14CFEnergy().
- Parameters
nb14_numbers (int32) – the number of necessary dihedral 1,4 terms M.
atom_numbers (int32) – the number of atoms N.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJ_type (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge of each atom.
boxlength_f (Tensor, float32) - [3,], the length of molecular simulation box in 3 dimensions.
a_14 (Tensor, int32) - [M,], the first atom index of each dihedral 1,4 term.
b_14 (Tensor, int32) - [M,], the second atom index of each dihedral 1,4 term.
lj_scale_factor (Tensor, float32) - [M,], the scale factor for the Lennard-Jones part of force correction of each dihedral 1,4 term.
cf_scale_factor (Tensor, float) - [M,], the scale factor for the Coulomb part of force correction for each dihedral 1,4 terms.
LJ_type_A (Tensor, float32) - [Q,], the A parameter in Lennard-Jones scheme of each atom pair type. Q is the number of atom pair.
LJ_type_B (Tensor, float32) - [Q,], the B parameter in Lennard-Jones shceme of each atom pair type. Q is the number of atom pair.
- Outputs:
frc_f (Tensor, float32) - [N, 3], the force felt by each atom.
atom_energy (Tensor, float32) - [N,], the accumulated potential energy for each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.Dihedral14LJEnergy(*args, **kwargs)[source]¶ Calculate the Lennard-Jones part of 1,4 dihedral energy correction for each necessary dihedral terms on the corresponding atoms.
\[dr = (x_a-x_b, y_a-y_b, z_a-z-b)\]\[E = k*(A/|dr|^{12} - B/|dr|^{6})\]- Parameters
nb14_numbers (int32) – the number of necessary dihedral 1,4 terms M.
atom_numbers (int32) – the number of atoms N.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJ_type (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge of each atom.
boxlength_f (Tensor, float32) - [3,], the length of molecular simulation box in 3 dimensions.
a_14 (Tensor, int32) - [M,], the first atom index of each dihedral 1,4 term.
b_14 (Tensor, int32) - [M,], the second atom index of each dihedral 1,4 term.
lj_scale_factor (Tensor, float32) - [M,], the scale factor for the Lennard-Jones part of force correction of each dihedral 1,4 term.
LJ_type_A (Tensor, float32) - [Q,], the A parameter in Lennard-Jones scheme of each atom pair type. Q is the number of atom pair.
LJ_type_B (Tensor, float32) - [Q,], the B parameter in Lennard-Jones shceme of each atom pair type. Q is the number of atom pair.
- Outputs:
ene (Tensor, float32) - [M,], the Lennard-Jones potential energy correction for each necessary dihedral 1,4 term.
- Supported Platforms:
GPU
-
class
tinyms.primitives.Dihedral14LJForce(*args, **kwargs)[source]¶ Calculate the Lennard-Jones part of 1,4 dihedral force correction for each necessary dihedral terms on the corresponding atoms.
Assume the number of necessary dihedral 1,4 terms is M, the number of atoms is N, and the number of Lennard-Jones types for all atoms is P, which means there will be Q = P*(P+1)/2 types of possible Lennard-Jones interactions for all kinds of atom pairs.
\[dr = (x_a-x_b, y_a-y_b, z_a-z_b)\]\[F = k*(-12*A/|dr|^{14} + 6*B/|dr|^{8})*dr\]- Parameters
nb14_numbers (int32) – the number of necessary dihedral 1,4 terms M.
atom_numbers (int32) – the number of atoms N.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJ_type (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge of each atom.
boxlength_f (Tensor, float32) - [3,], the length of molecular simulation box in 3 dimensions.
a_14 (Tensor, int32) - [M,], the first atom index of each dihedral 1,4 term.
b_14 (Tensor, int32) - [M,], the second atom index of each dihedral 1,4 term.
lj_scale_factor (Tensor, float32) - [M,], the scale factor for the Lennard-Jones part of force correction of each dihedral 1,4 term.
LJ_type_A (Tensor, float32) - [Q,], the A parameter in Lennard-Jones scheme of each atom pair type. Q is the number of atom pair.
LJ_type_B (Tensor, float32) - [Q,], the B parameter in Lennard-Jones shceme of each atom pair type. Q is the number of atom pair.
- Outputs:
frc_f (Tensor, float32) - [N, 3], the force felt by each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.Dihedral14LJForceWithDirectCF(*args, **kwargs)[source]¶ Calculate the Lennard-Jones part and the Coulomb part of force correction for each necessary dihedral 1,4 terms.
The calculation formula of the Lennard-Jones part is the same as operator Dihedral14LJForce(), and the Coulomb part is as follows:
\[dr = (x_a-x_b, y_a-y_b, z_a-z_b)\]\[F = -k*q_a*q_b/|r|^3*dr\]- Parameters
nb14_numbers (int32) – the number of necessary dihedral 1,4 terms M.
atom_numbers (int32) – the number of atoms N.
- Inputs:
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJ_type (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge of each atom.
boxlength_f (Tensor, float32) - [3,], the length of molecular simulation box in 3 dimensions.
a_14 (Tensor, int32) - [M,], the first atom index of each dihedral 1,4 term.
b_14 (Tensor, int32) - [M,], the second atom index of each dihedral 1,4 term.
lj_scale_factor (Tensor, float32) - [M,], the scale factor for the Lennard-Jones part of force correction of each dihedral 1,4 term.
cf_scale_factor (Tensor, float) - [M,], the scale factor for the Coulomb part of force correction for each dihedral 1,4 terms.
LJ_type_A (Tensor, float32) - [Q,], the A parameter in Lennard-Jones scheme of each atom pair type. Q is the number of atom pair.
LJ_type_B (Tensor, float32) - [Q,], the B parameter in Lennard-Jones shceme of each atom pair type. Q is the number of atom pair.
- Outputs:
frc_f (Tensor, float) - [N, 3], the force felt by each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.DihedralAtomEnergy(*args, **kwargs)[source]¶ Add the potential energy caused by dihedral terms to the total potential energy of each atom.
The calculation formula is the same as operator DihedralEnergy().
- Parameters
dihedral_numbers (int32) – the number of dihedral terms M.
- Inputs:
dihedral_numbers (int32) - the number of dihedral terms M.
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinates value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the 1st atom index of each dihedral.
atom_b (Tensor, int32) - [M,], the 2nd atom index of each dihedral.
atom_c (Tensor, int32) - [M,], the 3rd atom index of each dihedral.
atom_d (Tensor, int32) - [M,], the 4th atom index of each dihedral. 4 atoms are connected in the form a-b-c-d.
ipn (Tensor, int32) - [M,], the period of dihedral angle of each dihedral.
pk (Tensor, float32) - [M,], the force constant of each dihedral.
gamc (Tensor, float32) - [M,], k*cos(phi_0) of each dihedral.
gams (Tensor, float32) - [M,], k*sin(phi_0) of each dihedral.
pn (Tensor, float32) - [M,], the floating point form of ipn.
- Outputs:
ene (Tensor, float32) - [N,], the accumulated potential energy for each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.DihedralEnergy(*args, **kwargs)[source]¶ Calculate the potential energy caused by dihedral terms for each 4-atom pair. Assume our system has N atoms and M dihedral terms.
\[E = k(1 + cos(n*phi - phi_0))\]- Parameters
dihedral_numbers (int32) – the number of dihedral terms M.
- Inputs:
dihedral_numbers (int32) - the number of dihedral terms M.
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinates value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the 1st atom index of each dihedral.
atom_b (Tensor, int32) - [M,], the 2nd atom index of each dihedral.
atom_c (Tensor, int32) - [M,], the 3rd atom index of each dihedral.
atom_d (Tensor, int32) - [M,], the 4th atom index of each dihedral. 4 atoms are connected in the form a-b-c-d.
ipn (Tensor, int32) - [M,], the period of dihedral angle of each dihedral.
pk (Tensor, float32) - [M,], the force constant of each dihedral.
gamc (Tensor, float32) - [M,], k*cos(phi_0) of each dihedral.
gams (Tensor, float32) - [M,], k*sin(phi_0) of each dihedral.
pn (Tensor, float32) - [M,], the floating point form of ipn.
- Outputs:
ene (Tensor, float32) - [M,], the potential energy for each dihedral term.
- Supported Platforms:
GPU
-
class
tinyms.primitives.DihedralForce(*args, **kwargs)[source]¶ Calculate the force exerted by the dihedral term which made of 4-atoms on the corresponding atoms. Assume the number of dihedral terms is M and the number of atoms is N.
\[dr_{ab} = (x_b-x_a, y_b-y_a, z_b-z_a)\]\[dr_{cb} = (x_b-x_c, y_b-y_c, z_b-z_c)\]\[dr_{cd} = (x_d-x_c, y_d-y_c, z_d-z_c)\]\[r1 = dr_{ab}*dr_{cb}\]\[r2 = dr_{cd}*dr_{cb}\]\[phi = pi - sign(inner_product(r1*r2), dr_{cb}) * arccos(inner_product(r1, r2)/|r1|/|r2|)\]\[dEdphi = n*phi*(k*cos(phi_0)*sin(n*phi) - k*sin(phi_0)*cos(n*phi))/sin(phi)\]\[dphidr1 = r2/|r1|/|r2| + cos(phi)/|r1|^2*r1\]\[dphidr2 = r1/|r1|/|r2| + cos(phi)/|r2|^2*r2\]\[dEdra = dEdphi * dr_{cb} * dphidr1\]\[dEdrd = dEdphi * dphi_dr2 * dr_{cb}\]\[dEdrjpart = dEdphi * ((dr_{ab} * dphidr1) + (dr_{cd} * dphidr2))\]\[F_a = dEdri\]\[F_b = dEdrjpart - dEdri\]\[F_c = - dEdrl - dEdrjpart\]\[F_d = dEdrl\]- Parameters
dihedral_numbers (int32) – the number of dihedral terms M.
- Inputs:
dihedral_numbers (int32) - the number of dihedral terms M.
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinates value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the 1st atom index of each dihedral.
atom_b (Tensor, int32) - [M,], the 2nd atom index of each dihedral.
atom_c (Tensor, int32) - [M,], the 3rd atom index of each dihedral.
atom_d (Tensor, int32) - [M,], the 4th atom index of each dihedral. 4 atoms are connected in the form a-b-c-d.
ipn (Tensor, int32) - [M,], the period of dihedral angle of each dihedral.
pk (Tensor, float32) - [M,], the force constant of each dihedral.
gamc (Tensor, float32) - [M,], k*cos(phi_0) of each dihedral.
gams (Tensor, float32) - [M,], k*sin(phi_0) of each dihedral.
pn (Tensor, float32) - [M,], the floating point form of ipn.
- Outputs:
frc_f (Tensor, float32) - [N, 3], the force felt by each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.DihedralForceWithAtomEnergy(*args, **kwargs)[source]¶ Calculate dihedral force and potential energy together.
The calculation formula is the same as operator DihedralForce() and DihedralEnergy().
- Parameters
dihedral_numbers (int32) – the number of dihedral terms M.
- Inputs:
dihedral_numbers (int32) - the number of dihedral terms M.
uint_crd_f (Tensor, uint32) - [N, 3], the unsigned int coordinates value of each atom.
scaler_f (Tensor, float32) - [3,], the 3-D scale factor between the real space float coordinates and the unsigned int coordinates.
atom_a (Tensor, int32) - [M,], the 1st atom index of each dihedral.
atom_b (Tensor, int32) - [M,], the 2nd atom index of each dihedral.
atom_c (Tensor, int32) - [M,], the 3rd atom index of each dihedral.
atom_d (Tensor, int32) - [M,], the 4th atom index of each dihedral. 4 atoms are connected in the form a-b-c-d.
ipn (Tensor, int32) - [M,], the period of dihedral angle of each dihedral.
pk (Tensor, float32) - [M,], the force constant of each dihedral.
gamc (Tensor, float32) - [M,], k*cos(phi_0) of each dihedral.
gams (Tensor, float32) - [M,], k*sin(phi_0) of each dihedral.
pn (Tensor, float32) - [M,], the floating point form of ipn.
- Outputs:
frc_f (Tensor, float32) - [N, 3], same as operator DihedralForce().
ene (Tensor, float32) - [N,], same as operator DihedralAtomEnergy().
- Supported Platforms:
GPU
-
class
tinyms.primitives.Div(*args, **kwargs)[source]¶ Computes the quotient of dividing the first input tensor by the second input tensor element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
\[out_{i} = \frac{x_i}{y_i}\]- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - When the first input is a tensor, The second input could be a number, a bool, or a tensor whose data type is number or bool. When the first input is a number or a bool, the second input must be a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-4.0, 5.0, 6.0]), mindspore.float32) >>> input_y = Tensor(np.array([3.0, 2.0, 3.0]), mindspore.float32) >>> div = ops.Div() >>> output = div(input_x, input_y) >>> print(output) [-1.3333334 2.5 2. ]
-
class
tinyms.primitives.DivNoNan(*args, **kwargs)[source]¶ Computes a safe divide and returns 0 if the y is zero.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([-1.0, 0., 1.0, 5.0, 6.0]), mindspore.float32) >>> input_y = Tensor(np.array([0., 0., 0., 2.0, 3.0]), mindspore.float32) >>> div_no_nan = ops.DivNoNan() >>> output = div_no_nan(input_x, input_y) >>> print(output) [0. 0. 0. 2.5 2. ]
-
class
tinyms.primitives.Dropout(*args, **kwargs)[source]¶ During training, randomly zeroes some of the elements of the input tensor with probability.
- Parameters
- Inputs:
input (Tensor) - The input of Dropout with data type of float16 or float32.
- Outputs:
output (Tensor) - with the same shape as the input tensor.
mask (Tensor) - with the same shape as the input tensor.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> dropout = ops.Dropout(keep_prob=0.5) >>> x = Tensor((20, 16, 50, 50), mindspore.float32) >>> output, mask = dropout(x) >>> print(output) [0. 32. 0. 0.] >>> print(mask) [0. 1. 0. 0.]
-
class
tinyms.primitives.Dropout2D(*args, **kwargs)[source]¶ During training, randomly zeroes some of the channels of the input tensor with probability 1-keep_prob from a Bernoulli distribution.
- Parameters
keep_prob (float) – The keep probability of a channel, between 0 and 1, e.g. keep_prob = 0.8, means dropping out 20% of channels. Default: 0.5.
- Inputs:
input (Tensor) - A 4-D tensor with shape \((N, C, H, W)\).
- Outputs:
output (Tensor) - with the same shape and data type as the input tensor.
mask (Tensor[bool]) - with the same shape as the input tensor.
- Raises
TypeError – If the data type of keep_prob is not float.
ValueError – If keep_prob is out of the range [0.0, 1.0]; or if the dim of input is not 4-D.
- Supported Platforms:
Ascend
Examples
>>> dropout = ops.Dropout2D(keep_prob=0.5) >>> x = Tensor(np.random.randn(2, 1, 2, 3), mindspore.float32) >>> output, mask = dropout(x) >>> print(output) [[[[0. 0. 0.] [0. 0. 0.]]] [[[0.88 -2.98 -0.01] [2.16 -0.34 1.57]]]] >>> print(mask) [[[[False False False] [False False False]]] [[[True True True] [True True True]]]]
-
class
tinyms.primitives.Dropout3D(*args, **kwargs)[source]¶ During training, randomly zeroes some of the channels of the input tensor with probability 1-keep_prob from a Bernoulli distribution.
- Parameters
keep_prob (float) – The keep probability of a channel, between 0 and 1, e.g. keep_prob = 0.8, means dropping out 20% of channels. Default: 0.5.
- Inputs:
input (Tensor) - A 5-D tensor with shape \((N, C, D, H, W)\).
- Outputs:
output (Tensor) - with the same shape and data type as the input tensor.
mask (Tensor[bool]) - with the same shape as the input tensor.
- Raises
TypeError – If the data type of keep_prob is not float.
ValueError – If keep_prob is out of the range [0.0, 1.0]; or if the dim of input is not 5-D.
- Supported Platforms:
Ascend
Examples
>>> dropout = ops.Dropout3D(keep_prob=0.5) >>> x = Tensor(np.random.randn(2, 1, 2, 1, 2), mindspore.float32) >>> output, mask = dropout(x) >>> print(output) [[[[[0. 0.]] [[0. 0.]]]] [[[[-2.98 -0.01]] [[-0.34 1.57]]]]] >>> print(mask) [[[[[False False]] [[False False]]]] [[[[True True]] [[True True]]]]]
-
class
tinyms.primitives.DropoutDoMask(*args, **kwargs)[source]¶ Applies dropout mask on the input tensor.
Take the mask output of DropoutGenMask as input, and apply dropout on the input.
- Inputs:
input_x (Tensor) - The input tensor.
mask (Tensor) - The mask to be applied on input_x, which is the output of DropoutGenMask. And the shape of input_x must be the same as the value of DropoutGenMask’s input shape. If input wrong mask, the output of DropoutDoMask are unpredictable.
keep_prob (Union[Tensor, float]) - The keep rate, greater than 0 and less equal than 1, e.g. keep_prob = 0.9, means dropping out 10% of input units. The value of keep_prob is the same as the input keep_prob of DropoutGenMask.
- Outputs:
Tensor, the value that applied dropout on.
- Raises
TypeError – If input_x, mask or keep_prob is not a Tensor.
TypeError – If keep_prob is not a float.
ValueError – If value of keep_prob is not same as DropoutGenMaks.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.ones([2, 2, 3]), mindspore.float32) >>> shape = (2, 2, 3) >>> keep_prob = Tensor(0.5, mindspore.float32) >>> dropout_gen_mask = ops.DropoutGenMask() >>> dropout_do_mask = ops.DropoutDoMask() >>> mask = dropout_gen_mask(shape, keep_prob) >>> output = dropout_do_mask(x, mask, keep_prob) >>> print(output.shape) (2, 2, 3)
-
class
tinyms.primitives.DropoutGenMask(*args, **kwargs)[source]¶ Generates the mask value for the input shape.
- Parameters
- Inputs:
shape (tuple[int]) - The shape of target mask.
keep_prob (Tensor) - The keep rate, greater than 0 and less equal than 1, e.g. keep_prob = 0.9, means dropping out 10% of input units.
- Outputs:
Tensor, the value of generated mask for input shape.
- Raises
- Supported Platforms:
Ascend
Examples
>>> dropout_gen_mask = ops.DropoutGenMask() >>> shape = (2, 4, 5) >>> keep_prob = Tensor(0.5, mindspore.float32) >>> output = dropout_gen_mask(shape, keep_prob) >>> print(output.shape) (16,)
-
class
tinyms.primitives.DynamicGRUV2(*args, **kwargs)[source]¶ Applies a single-layer gated recurrent unit (GRU) to an input sequence.
\[\begin{split}\begin{array}{ll} r_t = \sigma(W_{ir} x_t + b_{ir} + W_{hr} h_{(t-1)} + b_{hr}) \\ z_t = \sigma(W_{iz} x_t + b_{iz} + W_{hz} h_{(t-1)} + b_{hz}) \\ n_t = \tanh(W_{in} x_t + b_{in} + r_t * (W_{hn} h_{(t-1)}+ b_{hn})) \\ h_t = (1 - z_t) * n_t + z_t * h_{(t-1)} \end{array}\end{split}\]where \(h_t\) is the hidden state at time t, \(x_t\) is the input at time t, \(h_{(t-1)}\) is the hidden state of the layer at time t-1 or the initial hidden state at time 0, and \(r_t\), \(z_t\), \(n_t\) are the reset, update, and new gates, respectively. \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product.
- Parameters
direction (str) – A string identifying the direction in the op. Default: ‘UNIDIRECTIONAL’. Only ‘UNIDIRECTIONAL’ is currently supported.
cell_depth (int) – An integer identifying the cell depth in the op. Default: 1.
keep_prob (float) – A float identifying the keep prob in the op. Default: 1.0.
cell_clip (float) – A float identifying the cell clip in the op. Default: -1.0.
num_proj (int) – An integer identifying the num proj in the op. Default: 0.
time_major (bool) – A bool identifying the time major in the op. Default: True.
activation (str) – A string identifying the type of activation function in the op. Default: ‘tanh’. Only ‘tanh’ is currently supported.
gate_order (str) – A string identifying the gate order in weight and bias. Default: ‘rzh. ‘zrh’ is another option.
reset_after (bool) – A bool identifying whether to apply reset gate after matrix multiplication. Default: True.
is_training (bool) – A bool identifying is training in the op. Default: True.
- Inputs:
x (Tensor) - Current words. Tensor of shape \((\text{num_step}, \text{batch_size}, \text{input_size})\). The data type must be float16.
weight_input (Tensor) - Input-hidden weight. Tensor of shape \((\text{input_size}, 3 \times \text{hidden_size})\). The data type must be float16.
weight_hidden (Tensor) - Hidden-hidden weight. Tensor of shape \((\text{hidden_size}, 3 \times \text{hidden_size})\). The data type must be float16.
bias_input (Tensor) - Input-hidden bias. Tensor of shape \((3 \times \text{hidden_size})\), or None. Has the same data type with input init_h.
bias_hidden (Tensor) - Hidden-hidden bias. Tensor of shape \((3 \times \text{hidden_size})\), or None. Has the same data type with input init_h.
seq_length (Tensor) - The length of each batch. Tensor of shape \((\text{batch_size})\). Only None is currently supported.
init_h (Tensor) - Hidden state of initial time. Tensor of shape \((\text{batch_size}, \text{hidden_size})\). The data type must be float16 or float32.
- Outputs:
y (Tensor) - A Tensor of shape :math: if num_proj > 0 (num_step, batch_size, min(hidden_size, num_proj), if num_proj == 0 (num_step, batch_size, hidden_size). Has the same data type with input bias_type.
output_h (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
update (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
reset (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
new (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
hidden_new (Tensor) - A Tensor of shape \((\text{num_step}, \text{batch_size}, \text{hidden_size})\). Has the same data type with input bias_type.
A note about the bias_type:
If bias_input and bias_hidden both are None, bias_type is date type of init_h.
If bias_input is not None, bias_type is the date type of bias_input.
If bias_input is None and bias_hidden is not None, `bias_type is the date type of bias_hidden.
- Raises
TypeError – If direction, activation or gate_order is not a str.
TypeError – If cell_depth or num_proj is not an int.
TypeError – If keep_prob or cell_clip is not a float.
TypeError – If time_major, reset_after or is_training is not a bool.
TypeError – If x, weight_input, weight_hidden, bias_input, bias_hidden, seq_length or ini_h is not a Tensor.
TypeError – If dtype of x, weight_input or weight_hidden is not float16.
TypeError – If dtype of init_h is neither float16 nor float32.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(2, 8, 64).astype(np.float16)) >>> weight_i = Tensor(np.random.rand(64, 48).astype(np.float16)) >>> weight_h = Tensor(np.random.rand(16, 48).astype(np.float16)) >>> bias_i = Tensor(np.random.rand(48).astype(np.float16)) >>> bias_h = Tensor(np.random.rand(48).astype(np.float16)) >>> init_h = Tensor(np.random.rand(8, 16).astype(np.float16)) >>> dynamic_gru_v2 = ops.DynamicGRUV2() >>> output = dynamic_gru_v2(x, weight_i, weight_h, bias_i, bias_h, None, init_h) >>> print(output[0].shape) (2, 8, 16)
-
class
tinyms.primitives.DynamicRNN(*args, **kwargs)[source]¶ Applies a recurrent neural network to the input. Only long short-term memory (LSTM) currently supported.
\[\begin{split}\begin{array}{ll} \\ i_t = \sigma(W_{ix} x_t + b_{ix} + W_{ih} h_{(t-1)} + b_{ih}) \\ f_t = \sigma(W_{fx} x_t + b_{fx} + W_{fh} h_{(t-1)} + b_{fh}) \\ \tilde{c}_t = \tanh(W_{cx} x_t + b_{cx} + W_{ch} h_{(t-1)} + b_{ch}) \\ o_t = \sigma(W_{ox} x_t + b_{ox} + W_{oh} h_{(t-1)} + b_{oh}) \\ c_t = f_t * c_{(t-1)} + i_t * \tilde{c}_t \\ h_t = o_t * \tanh(c_t) \\ \end{array}\end{split}\]Here \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product. \(W, b\) are learnable weights between the output and the input in the formula. For instance, \(W_{ix}, b_{ix}\) are the weight and bias used to transform from input \(x\) to \(i\).
- Parameters
cell_type (str) – A string identifying the cell type in the op. Default: ‘LSTM’. Only ‘LSTM’ is currently supported.
direction (str) – A string identifying the direction in the op. Default: ‘UNIDIRECTIONAL’. Only ‘UNIDIRECTIONAL’ is currently supported.
cell_depth (int) – An integer identifying the cell depth in the op. Default: 1.
use_peephole (bool) – A bool identifying if use peephole in the op. Default: False.
keep_prob (float) – A float identifying the keep prob in the op. Default: 1.0.
cell_clip (float) – A float identifying the cell clip in the op. Default: -1.0.
num_proj (int) – An integer identifying the num proj in the op. Default: 0.
time_major (bool) – A bool identifying the time major in the op. Default: True. Only True is currently supported.
activation (str) – A string identifying the type of activation function in the op. Default: ‘tanh’. Only ‘tanh’ is currently supported.
forget_bias (float) – A float identifying the forget bias in the op. Default: 0.0.
is_training (bool) – A bool identifying is training in the op. Default: True.
- Inputs:
x (Tensor) - Current words. Tensor of shape (num_step, batch_size, input_size). The data type must be float16.
w (Tensor) - Weight. Tensor of shape (input_size + hidden_size, 4 x hidden_size). The data type must be float16.
b (Tensor) - Bias. Tensor of shape (4 x hidden_size). The data type must be float16 or float32.
seq_length (Tensor) - The length of each batch. Tensor of shape (batch_size). Only None is currently supported.
init_h (Tensor) - Hidden state of initial time. Tensor of shape (1, batch_size, hidden_size). The data type must be float16.
init_c (Tensor) - Cell state of initial time. Tensor of shape (1, batch_size, hidden_size). The data type must be float16.
- Outputs:
y (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
output_h (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). With data type of float16.
output_c (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
i (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
j (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
f (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
o (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
tanhct (Tensor) - A Tensor of shape (num_step, batch_size, hidden_size). Has the same type with input b.
- Raises
TypeError – If cell_type, direction or activation is not a str.
TypeError – If cell_depth or num_proj is not an int.
TypeError – If keep_prob, cell_clip or forget_bias is not a float.
TypeError – If use_peehpole, time_major or is_training is not a bool.
TypeError – If x, w, b, seq_length, init_h or init_c is not a Tensor.
TypeError – If dtype of x, w, init_h or nit_c is not float16.
TypeError – If dtype of b is neither float16 nor float32.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(2, 16, 64).astype(np.float16)) >>> w = Tensor(np.random.rand(96, 128).astype(np.float16)) >>> b = Tensor(np.random.rand(128).astype(np.float16)) >>> init_h = Tensor(np.random.rand(1, 16, 32).astype(np.float16)) >>> init_c = Tensor(np.random.rand(1, 16, 32).astype(np.float16)) >>> dynamic_rnn = ops.DynamicRNN() >>> output = dynamic_rnn(x, w, b, None, init_h, init_c) >>> print(output[0].shape) (2, 16, 32)
-
class
tinyms.primitives.DynamicShape(*args, **kwargs)[source]¶ Returns the shape of the input tensor.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor[int], 1-dim Tensor of type int32
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) >>> shape = ops.DynamicShape() >>> output = shape(input_tensor) >>> print(output) [3 2 1]
-
class
tinyms.primitives.EditDistance(*args, **kwargs)[source]¶ Computes the Levenshtein Edit Distance. It is used to measure the similarity of two sequences. The inputs are variable-length sequences provided by SparseTensors (hypothesis_indices, hypothesis_values, hypothesis_shape) and (truth_indices, truth_values, truth_shape).
- Parameters
normalize (bool) – If true, edit distances are normalized by length of truth. Default: True.
- Inputs:
hypothesis_indices (Tensor) - The indices of the hypothesis list SparseTensor. With int64 data type. The shape of tensor is \((N, R)\).
hypothesis_values (Tensor) - The values of the hypothesis list SparseTensor. Must be 1-D vector with length of N.
hypothesis_shape (Tensor) - The shape of the hypothesis list SparseTensor. Must be R-length vector with int64 data type. Only constant value is allowed.
truth_indices (Tensor) - The indices of the truth list SparseTensor. With int64 data type. The shape of tensor is \((M, R)\).
truth_values (Tensor) - The values of the truth list SparseTensor. Must be 1-D vector with length of M.
truth_shape (Tensor) - The shape of the truth list SparseTensor. Must be R-length vector with int64 data type. Only constant value is allowed.
- Outputs:
Tensor, a dense tensor with rank R-1 and float32 data type.
- Raises
TypeError – If normalize is not a bool.
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> from mindspore import context >>> from mindspore import Tensor >>> import mindspore.nn as nn >>> import mindspore.ops.operations as ops >>> class EditDistance(nn.Cell): ... def __init__(self, hypothesis_shape, truth_shape, normalize=True): ... super(EditDistance, self).__init__() ... self.edit_distance = ops.EditDistance(normalize) ... self.hypothesis_shape = hypothesis_shape ... self.truth_shape = truth_shape ... ... def construct(self, hypothesis_indices, hypothesis_values, truth_indices, truth_values): ... return self.edit_distance(hypothesis_indices, hypothesis_values, self.hypothesis_shape, ... truth_indices, truth_values, self.truth_shape) ... >>> hypothesis_indices = Tensor(np.array([[0, 0, 0], [1, 0, 1], [1, 1, 1]]).astype(np.int64)) >>> hypothesis_values = Tensor(np.array([1, 2, 3]).astype(np.float32)) >>> hypothesis_shape = Tensor(np.array([1, 1, 2]).astype(np.int64)) >>> truth_indices = Tensor(np.array([[0, 1, 0], [0, 0, 1], [1, 1, 0], [1, 0, 1]]).astype(np.int64)) >>> truth_values = Tensor(np.array([1, 3, 2, 1]).astype(np.float32)) >>> truth_shape = Tensor(np.array([2, 2, 2]).astype(np.int64)) >>> edit_distance = EditDistance(hypothesis_shape, truth_shape) >>> output = edit_distance(hypothesis_indices, hypothesis_values, truth_indices, truth_values) >>> print(output) [[1. 1.] [1. 1.]]
-
class
tinyms.primitives.Elu(*args, **kwargs)[source]¶ Computes exponential linear:
\[\begin{split}\text{ELU}(x)= \left\{ \begin{array}{align} \alpha(e^{x} - 1) & \text{if } x \le 0\\ x & \text{if } x \gt 0\\ \end{array}\right.\end{split}\]The data type of input tensor must be float.
- Parameters
alpha (float) – The coefficient of negative factor whose type is float, only support ‘1.0’ currently. Default: 1.0.
- Inputs:
input_x (Tensor) - The input of Elu with data type of float16 or float32.
- Outputs:
Tensor, has the same shape and data type as input_x.
- Raises
TypeError – If alpha is not a float.
TypeError – If dtype of input_x is neither float16 nor float32.
ValueError – If alpha is not equal to 1.0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> elu = ops.Elu() >>> output = elu(input_x) >>> print(output) [[-0.63212055 4. -0.99966455] [ 2. -0.99326205 9. ]]
-
class
tinyms.primitives.EmbeddingLookup(*args, **kwargs)[source]¶ Returns a slice of input tensor based on the specified indices.
This Primitive has the similar functionality as GatherV2 operating on axis = 0, but has one more inputs: offset.
- Inputs:
input_params (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). This represents a Tensor slice, instead of the entire Tensor. Currently, the dimension is restricted to be 2.
input_indices (Tensor) - The shape of tensor is \((y_1, y_2, ..., y_S)\). Specifies the indices of elements of the original Tensor. Values can be out of range of input_params, and the exceeding part will be filled with 0 in the output. Values does not support negative and the result is undefined if values are negative.
offset (int) - Specifies the offset value of this input_params slice. Thus the real indices are equal to input_indices minus offset.
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Raises
TypeError – If dtype of input_indices is not int.
ValueError – If length of shape of input_params is greater than 2.
- Supported Platforms:
AscendCPU
Examples
>>> input_params = Tensor(np.array([[8, 9], [10, 11], [12, 13], [14, 15]]), mindspore.float32) >>> input_indices = Tensor(np.array([[5, 2], [8, 5]]), mindspore.int32) >>> offset = 4 >>> output = ops.EmbeddingLookup()(input_params, input_indices, offset) >>> print(output) [[[10. 11.] [ 0. 0.]] [[ 0. 0.] [10. 11.]]]
-
class
tinyms.primitives.Eps(*args, **kwargs)[source]¶ Creates a tensor filled with input_x dtype minimum value.
- Inputs:
input_x (Tensor) - Input tensor. The data type must be float16 or float32.
- Outputs:
Tensor, has the same type and shape as input_x, but filled with input_x dtype minimum val.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor([4, 1, 2, 3], mindspore.float32) >>> output = ops.Eps()(input_x) >>> print(output) [1.5258789e-05 1.5258789e-05 1.5258789e-05 1.5258789e-05]
-
class
tinyms.primitives.Equal(*args, **kwargs)[source]¶ Computes the equivalence between two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number]) - The first input is a number or a tensor whose data type is number.
input_y (Union[Tensor, Number]) - The second input is a number when the first input is a tensor or a tensor whose data type is number. The data type is the same as the first input.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> equal = ops.Equal() >>> output = equal(input_x, 2.0) >>> print(output) [False True False] >>> >>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 2, 4]), mindspore.int32) >>> equal = ops.Equal() >>> output = equal(input_x, input_y) >>> print(output) [ True True False]
-
class
tinyms.primitives.EqualCount(*args, **kwargs)[source]¶ Computes the number of the same elements of two tensors.
The two input tensors must have the same data type and shape.
- Inputs:
input_x (Tensor) - The first input tensor.
input_y (Tensor) - The second input tensor.
- Outputs:
Tensor, with the type same as input tensor and size as (1,).
- Raises
TypeError – If input_x or input_y is not a Tensor.
ValueError – If shape of input_x is not equal to shape of input_y.
- Supported Platforms:
GPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 2, 4]), mindspore.int32) >>> equal_count = ops.EqualCount() >>> output = equal_count(input_x, input_y) >>> print(output) [2]
-
class
tinyms.primitives.Erf(*args, **kwargs)[source]¶ Computes the Gauss error function of input_x element-wise.
\[erf(x)=\frac{2} {\sqrt{\pi}} \int\limits_0^{x} e^{-t^{2}} dt\]- Inputs:
input_x (Tensor) - The input tensor. The data type must be float16 or float32.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Raises
TypeError – If dtype of input_x is neither float16 nor float32.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([-1, 0, 1, 2, 3]), mindspore.float32) >>> erf = ops.Erf() >>> output = erf(input_x) >>> print(output) [-0.8427168 0. 0.8427168 0.99530876 0.99997765]
-
class
tinyms.primitives.Erfc(*args, **kwargs)[source]¶ Computes the complementary error function of input_x element-wise.
\[erfc(x) = 1 - \frac{2} {\sqrt{\pi}} \int\limits_0^{x} e^{-t^{2}} dt\]- Inputs:
input_x (Tensor) - The input tensor. The data type must be float16 or float32.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Raises
TypeError – If dtype of input_x is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([-1, 0, 1, 2, 3]), mindspore.float32) >>> erfc = ops.Erfc() >>> output = erfc(input_x) >>> print(output) [1.8427168e+00 1.0000000e+00 1.5728319e-01 4.6912432e-03 2.2351742e-05]
-
class
tinyms.primitives.Evolution(*args, **kwargs)[source]¶ Inputs of this operation is generated by MindQuantum framework.
- Inputs:
n_qubits (int) - The qubit number of quantum simulator.
param_names (List[str]) - The parameters names.
gate_names (List[str]) - The name of each gate.
- gate_matrix (List[List[List[List[float]]]]) - Real part and image
part of the matrix of quantum gate.
gate_obj_qubits (List[List[int]]) - Object qubits of each gate.
gate_ctrl_qubits (List[List[int]]) - Control qubits of each gate.
gate_params_names (List[List[str]]) - Parameter names of each gate.
gate_coeff (List[List[float]]) - Coefficient of eqch parameter of each gate.
- gate_requires_grad (List[List[bool]]) - Whether to calculate gradient
of parameters of gates.
hams_pauli_coeff (List[List[float]]) - Coefficient of pauli words.
hams_pauli_word (List[List[List[str]]]) - Pauli words.
hams_pauli_qubit (List[List[List[int]]]) - The qubit that pauli matrix act on.
- Outputs:
Quantum state (Tensor) - The quantum state after evolution.
- Supported Platforms:
CPU
-
class
tinyms.primitives.Exp(*args, **kwargs)[source]¶ Returns exponential of a tensor element-wise.
\[out_i = e^{x_i}\]- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> exp = ops.Exp() >>> output = exp(input_x) >>> print(output) [ 2.718282 7.389056 54.598152]
-
class
tinyms.primitives.ExpandDims(*args, **kwargs)[source]¶ Adds an additional dimension at the given axis.
Note
If the specified axis is a negative number, the index is counted backward from the end and starts at 1.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
axis (int) - Specifies the dimension index at which to expand the shape of input_x. The value of axis must be in the range [-input_x.ndim-1, input_x.ndim]. Only constant value is allowed.
- Outputs:
Tensor, the shape of tensor is \((1, x_1, x_2, ..., x_R)\) if the value of axis is 0. It has the same type as input_x.
- Raises
ValueError – If axis is not an int or not in the valid range.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> expand_dims = ops.ExpandDims() >>> output = expand_dims(input_tensor, 0) >>> print(output) [[[2. 2.] [2. 2.]]]
-
class
tinyms.primitives.Expm1(*args, **kwargs)[source]¶ Returns exponential then minus 1 of a tensor element-wise.
\[out_i = e^{x_i} - 1\]- Inputs:
input_x (Tensor) - The input tensor. With float16 or float32 data type.
- Outputs:
Tensor, has the same shape as the input_x.
- Raises
TypeError – If dtype of input_x is neither float16 nor float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([0.0, 1.0, 2.0, 4.0]), mindspore.float32) >>> expm1 = ops.Expm1() >>> output = expm1(input_x) >>> print(output) [ 0. 1.718282 6.389056 53.598152]
-
class
tinyms.primitives.Eye(*args, **kwargs)[source]¶ Creates a tensor with ones on the diagonal and zeros the rest.
- Inputs:
n (int) - The number of rows of returned tensor
m (int) - The number of columns of returned tensor
t (mindspore.dtype) - MindSpore’s dtype, The data type of the returned tensor.
- Outputs:
Tensor, a tensor with ones on the diagonal and the rest of elements are zero.
- Raises
TypeError – If m or n is not an int.
ValueError – If m or n is less than 1.
- Supported Platforms:
AscendGPUCPU
Examples
>>> eye = ops.Eye() >>> output = eye(2, 2, mindspore.int32) >>> print(output) [[1 0] [0 1]]
-
class
tinyms.primitives.FastGeLU(*args, **kwargs)[source]¶ Fast Gaussian Error Linear Units activation function.
FastGeLU is defined as follows:
\[\text{output} = \frac {x} {1 + \exp(-1.702 * \left| x \right|)} * \exp(0.851 * (x - \left| x \right|)),\]where \(x\) is the element of the input.
- Inputs:
input_x (Tensor) - Input to compute the FastGeLU with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as input.
- Raises
TypeError – If dtype of input_x is neither float16 nor float32.
- Supported Platforms:
Ascend
Examples
>>> tensor = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> fast_gelu = P.FastGeLU() >>> output = fast_gelu(tensor) >>> print(output) [[-1.5420423e-01 3.9955849e+00 -9.7664278e-06] [ 1.9356585e+00 -1.0070159e-03 8.9999981e+00]]
-
class
tinyms.primitives.FastGelu(**kwargs)[source]¶ Same as operator FastGeLU. FastGelu will be deprecated in the future. Please use FastGeLU instead.
-
class
tinyms.primitives.Fill(*args, **kwargs)[source]¶ Creates a tensor filled with a scalar value.
Creates a tensor with shape described by the first argument and fills it with values in the second argument.
- Inputs:
type (mindspore.dtype) - The specified type of output tensor. Only constant value is allowed.
shape (tuple) - The specified shape of output tensor. Only constant value is allowed.
value (scalar) - Value to fill the returned tensor. Only constant value is allowed.
- Outputs:
Tensor, has the same type and shape as input value.
- Raises
TypeError – If shape is not a tuple.
- Supported Platforms:
AscendGPUCPU
Examples
>>> fill = ops.Fill() >>> output = fill(mindspore.float32, (2, 2), 1) >>> print(output) [[1. 1.] [1. 1.]]
-
class
tinyms.primitives.Flatten(*args, **kwargs)[source]¶ Flattens a tensor without changing its batch size on the 0-th axis.
- Inputs:
input_x (Tensor) - Tensor of shape \((N, \ldots)\) to be flattened.
- Outputs:
Tensor, the shape of the output tensor is \((N, X)\), where \(X\) is the product of the remaining dimension.
- Raises
TypeError – If input_x is not a Tensor.
ValueError – If length of shape of input_x is less than 1.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.ones(shape=[1, 2, 3, 4]), mindspore.float32) >>> flatten = ops.Flatten() >>> output = flatten(input_tensor) >>> print(output.shape) (1, 24)
-
class
tinyms.primitives.FloatStatus(*args, **kwargs)[source]¶ Determines if the elements contain Not a Number(NaN), infinite or negative infinite. 0 for normal, 1 for overflow.
- Inputs:
input_x (Tensor) - The input tensor. The data type must be float16 or float32.
- Outputs:
Tensor, has the shape of (1,), and the dtype is mindspore.dtype.float32.
- Raises
TypeError – If dtype of input_x is neither float16 nor float32.
- Supported Platforms:
GPU
Examples
>>> float_status = ops.FloatStatus() >>> input_x = Tensor(np.array([np.log(-1), 1, np.log(0)]), mindspore.float32) >>> result = float_status(input_x) >>> print(result) [1.]
-
class
tinyms.primitives.Floor(*args, **kwargs)[source]¶ Rounds a tensor down to the closest integer element-wise.
\[out_i = \lfloor x_i \rfloor\]- Inputs:
input_x (Tensor) - The input tensor. Its element data type must be float.
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If dtype of input_x is not float.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.1, 2.5, -1.5]), mindspore.float32) >>> floor = ops.Floor() >>> output = floor(input_x) >>> print(output) [ 1. 2. -2.]
-
class
tinyms.primitives.FloorDiv(*args, **kwargs)[source]¶ Divides the first input tensor by the second input tensor element-wise and round down to the closest integer.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.int32) >>> input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) >>> floor_div = ops.FloorDiv() >>> output = floor_div(input_x, input_y) >>> print(output) [ 0 1 -1]
-
class
tinyms.primitives.FloorMod(*args, **kwargs)[source]¶ Computes the remainder of division element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool , and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.int32) >>> input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) >>> floor_mod = ops.FloorMod() >>> output = floor_mod(input_x, input_y) >>> print(output) [2 1 2]
-
class
tinyms.primitives.FusedSparseAdam(*args, **kwargs)[source]¶ Merges the duplicate value of the gradient and then updates parameters by the Adaptive Moment Estimation (Adam) algorithm. This operator is used when the gradient is sparse.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector, \(v\) represents the 2nd moment vector, \(g\) represents gradient, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents var, \(\epsilon\) represents epsilon.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
- Inputs:
var (Parameter) - Parameters to be updated with float32 data type.
m (Parameter) - The 1st moment vector in the updating formula, has the same type as var with float32 data type.
v (Parameter) - The 2nd moment vector in the updating formula. Mean square gradients, has the same type as var with float32 data type.
beta1_power (Tensor) - \(beta_1^t\) in the updating formula with float32 data type.
beta2_power (Tensor) - \(beta_2^t\) in the updating formula with float32 data type.
lr (Tensor) - \(l\) in the updating formula. With float32 data type.
beta1 (Tensor) - The exponential decay rate for the 1st moment estimations with float32 data type.
beta2 (Tensor) - The exponential decay rate for the 2nd moment estimations with float32 data type.
epsilon (Tensor) - Term added to the denominator to improve numerical stability with float32 data type.
gradient (Tensor) - Gradient value with float32 data type.
indices (Tensor) - Gradient indices with int32 data type.
- Outputs:
Tuple of 3 Tensors, this operator will update the input parameters directly, the outputs are useless.
var (Tensor) - A Tensor with shape (1,).
m (Tensor) - A Tensor with shape (1,).
v (Tensor) - A Tensor with shape (1,).
- Raises
- Supported Platforms:
AscendCPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_adam = ops.FusedSparseAdam() ... self.var = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="m") ... self.v = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="v") ... def construct(self, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad, indices): ... out = self.sparse_apply_adam(self.var, self.m, self.v, beta1_power, beta2_power, lr, beta1, beta2, ... epsilon, grad, indices) ... return out ... >>> net = Net() >>> beta1_power = Tensor(0.9, mstype.float32) >>> beta2_power = Tensor(0.999, mstype.float32) >>> lr = Tensor(0.001, mstype.float32) >>> beta1 = Tensor(0.9, mstype.float32) >>> beta2 = Tensor(0.999, mstype.float32) >>> epsilon = Tensor(1e-8, mstype.float32) >>> gradient = Tensor(np.random.rand(2, 1, 2), mstype.float32) >>> indices = Tensor([0, 1], mstype.int32) >>> output = net(beta1_power, beta2_power, lr, beta1, beta2, epsilon, gradient, indices) >>> print(net.var.asnumpy()) [[[0.9996963 0.9996977 ]] [[0.99970144 0.9996992 ]] [[0.99971527 0.99971527]]]
-
class
tinyms.primitives.FusedSparseFtrl(*args, **kwargs)[source]¶ Merges the duplicate value of the gradient and then updates relevant entries according to the FTRL-proximal scheme.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
lr (float) – The learning rate value, must be positive.
l1 (float) – l1 regularization strength, must be greater than or equal to zero.
l2 (float) – l2 regularization strength, must be greater than or equal to zero.
lr_power (float) – Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero.
use_locking (bool) – Use locks for updating operation if true . Default: False.
- Inputs:
var (Parameter) - The variable to be updated. The data type must be float32.
accum (Parameter) - The accumulation to be updated, must be same type and shape as var.
linear (Parameter) - the linear coefficient to be updated, must be same type and shape as var.
grad (Tensor) - A tensor of the same type as var, for the gradient.
indices (Tensor) - A vector of indices into the first dimension of var and accum. The shape of indices must be the same as grad in first dimension. The type must be int32.
- Outputs:
Tuple of 3 Tensor, this operator will update the input parameters directly, the outputs are useless.
var (Tensor) - A Tensor with shape (1,).
accum (Tensor) - A Tensor with shape (1,).
linear (Tensor) - A Tensor with shape (1,).
- Raises
TypeError – If lr, l1, l2 or lr_power is not a float.
ValueError – If shape of lr_power less than or equal to zero.
TypeError – If dtype of var is not float32.
TypeError – If dtype of indices is not int32.
TypeError – If shape of accum, linear or grad is not same as var.
TypeError – If shape of indices is not same as shape of first dimension of grad.
- Supported Platforms:
AscendCPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class SparseApplyFtrlNet(nn.Cell): ... def __init__(self): ... super(SparseApplyFtrlNet, self).__init__() ... self.sparse_apply_ftrl = ops.FusedSparseFtrl(lr=0.01, l1=0.0, l2=0.0, lr_power=-0.5) ... self.var = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="accum") ... self.linear = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="linear") ... ... def construct(self, grad, indices): ... out = self.sparse_apply_ftrl(self.var, self.accum, self.linear, grad, indices) ... return out ... >>> net = SparseApplyFtrlNet() >>> grad = Tensor(np.random.rand(2, 1, 2).astype(np.float32)) >>> indices = Tensor(np.array([0, 1]).astype(np.int32)) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]), Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]), Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]))
-
class
tinyms.primitives.FusedSparseLazyAdam(*args, **kwargs)[source]¶ Merges the duplicate value of the gradient and then updates parameters by the Adaptive Moment Estimation (LazyAdam) algorithm. This operator is used when the gradient is sparse. The behavior is not equivalent to the original Adam algorithm, as only the current indices parameters will be updated.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector, \(v\) represents the 2nd moment vector, \(g\) represents gradient, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents var, \(\epsilon\) represents epsilon.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
- Inputs:
var (Parameter) - Parameters to be updated with float32 data type.
m (Parameter) - The 1st moment vector in the updating formula, has the same type as var with float32 data type.
v (Parameter) - The 2nd moment vector in the updating formula. Mean square gradients, has the same type as var with float32 data type.
beta1_power (Tensor) - \(beta_1^t\) in the updating formula with float32 data type.
beta2_power (Tensor) - \(beta_2^t\) in the updating formula with float32 data type.
lr (Tensor) - \(l\) in the updating formula with float32 data type.
beta1 (Tensor) - The exponential decay rate for the 1st moment estimations with float32 data type.
beta2 (Tensor) - The exponential decay rate for the 2nd moment estimations with float32 data type.
epsilon (Tensor) - Term added to the denominator to improve numerical stability with float32 data type.
gradient (Tensor) - Gradient value with float32 data type.
indices (Tensor) - Gradient indices with int32 data type.
- Outputs:
Tuple of 3 Tensors, this operator will update the input parameters directly, the outputs are useless.
var (Tensor) - A Tensor with shape (1,).
m (Tensor) - A Tensor with shape (1,).
v (Tensor) - A Tensor with shape (1,).
- Raises
- Supported Platforms:
AscendCPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_lazyadam = ops.FusedSparseLazyAdam() ... self.var = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="var") ... self.m = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="m") ... self.v = Parameter(Tensor(np.ones([3, 1, 2]).astype(np.float32)), name="v") ... def construct(self, beta1_power, beta2_power, lr, beta1, beta2, epsilon, grad, indices): ... out = self.sparse_apply_lazyadam(self.var, self.m, self.v, beta1_power, beta2_power, lr, beta1, ... beta2, epsilon, grad, indices) ... return out ... >>> net = Net() >>> beta1_power = Tensor(0.9, mstype.float32) >>> beta2_power = Tensor(0.999, mstype.float32) >>> lr = Tensor(0.001, mstype.float32) >>> beta1 = Tensor(0.9, mstype.float32) >>> beta2 = Tensor(0.999, mstype.float32) >>> epsilon = Tensor(1e-8, mstype.float32) >>> gradient = Tensor(np.random.rand(2, 1, 2), mstype.float32) >>> indices = Tensor([0, 1], mstype.int32) >>> output = net(beta1_power, beta2_power, lr, beta1, beta2, epsilon, gradient, indices) >>> print(net.var.asnumpy()) [[[0.9996866 0.9997078]] [[0.9997037 0.9996869]] [[1. 1. ]]]
-
class
tinyms.primitives.FusedSparseProximalAdagrad(*args, **kwargs)[source]¶ Merges the duplicate value of the gradient and then updates relevant entries according to the proximal adagrad algorithm.
\[accum += grad * grad\]\[\text{prox_v} = var - lr * grad * \frac{1}{\sqrt{accum}}\]\[var = \frac{sign(\text{prox_v})}{1 + lr * l2} * \max(\left| \text{prox_v} \right| - lr * l1, 0)\]All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – If true, the variable and accumulation tensors will be protected from being updated. Default: False.
- Inputs:
var (Parameter) - Variable tensor to be updated. The data type must be float32.
accum (Parameter) - Variable tensor to be updated, has the same dtype as var.
lr (Tensor) - The learning rate value. The data type must be float32.
l1 (Tensor) - l1 regularization strength. The data type must be float32.
l2 (Tensor) - l2 regularization strength. The data type must be float32.
grad (Tensor) - A tensor of the same type as var, for the gradient. The data type must be float32.
indices (Tensor) - A vector of indices into the first dimension of var and accum. The data type must be int32.
- Outputs:
Tuple of 2 Tensors, this operator will update the input parameters directly, the outputs are useless.
var (Tensor) - A Tensor with shape (1,).
accum (Tensor) - A Tensor with shape (1,).
- Raises
- Supported Platforms:
CPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> import mindspore.common.dtype as mstype >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_proximal_adagrad = ops.FusedSparseProximalAdagrad() ... self.var = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(3, 1, 2).astype(np.float32)), name="accum") ... self.lr = Tensor(0.01, mstype.float32) ... self.l1 = Tensor(0.0, mstype.float32) ... self.l2 = Tensor(0.0, mstype.float32) ... def construct(self, grad, indices): ... out = self.sparse_apply_proximal_adagrad(self.var, self.accum, self.lr, self.l1, ... self.l2, grad, indices) ... return out ... >>> net = Net() >>> grad = Tensor(np.random.rand(2, 1, 2).astype(np.float32)) >>> indices = Tensor(np.array([0, 1]).astype(np.int32)) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]), Tensor(shape=[1], dtype=Float32, value= [0.00000000e+00]))
-
class
tinyms.primitives.Gamma(*args, **kwargs)[source]¶ Produces random positive floating-point values x, distributed according to probability density function:
\[\text{P}(x|α,β) = \frac{\exp(-x/β)}{{β^α}\cdot{\Gamma(α)}}\cdot{x^{α-1}},\]- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
alpha (Tensor) - The α distribution parameter. It must be greater than 0. It is also known as the shape parameter with float32 data type.
beta (Tensor) - The β distribution parameter. It must be greater than 0. It is also known as the scale parameter with float32 data type.
- Outputs:
Tensor. The shape must be the broadcasted shape of Input “shape” and shapes of alpha and beta. The dtype is float32.
- Raises
TypeError – If neither seed nor seed2 is an int.
TypeError – If neither alpha nor beta is a Tensor.
ValueError – If shape is not a constant value.
- Supported Platforms:
Ascend
Examples
>>> shape = (3, 1, 2) >>> alpha = Tensor(np.array([[3, 4], [5, 6]]), mstype.float32) >>> beta = Tensor(np.array([1.0]), mstype.float32) >>> gamma = ops.Gamma(seed=3) >>> output = gamma(shape, alpha, beta) >>> result = output.shape >>> print(result) (3, 2, 2)
-
class
tinyms.primitives.Gather(*args, **kwargs)[source]¶ Returns a slice of the input tensor based on the specified indices and axis.
Slices the input tensor base on the indices at specified axis. See the following example for more clear.
- Inputs:
input_params (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The original Tensor.
input_indices (Tensor) - The shape of tensor is \((y_1, y_2, ..., y_S)\). Specifies the indices of elements of the original Tensor. Must be in the range [0, input_param.shape[axis]).
axis (int) - Specifies the dimension index to gather indices.
- Outputs:
Tensor, the shape of tensor is \(input\_params.shape[:axis] + input\_indices.shape + input\_params.shape[axis + 1:]\).
- Raises
TypeError – If axis is not an int.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_params = Tensor(np.array([[1, 2, 7, 42], [3, 4, 54, 22], [2, 2, 55, 3]]), mindspore.float32) >>> input_indices = Tensor(np.array([1, 2]), mindspore.int32) >>> axis = 1 >>> output = ops.Gather()(input_params, input_indices, axis) >>> print(output) [[ 2. 7.] [ 4. 54.] [ 2. 55.]] >>> axis = 0 >>> output = ops.Gather()(input_params, input_indices, axis) >>> print(output) [[3. 4. 54. 22.] [2. 2. 55. 3.]]
-
class
tinyms.primitives.GatherD(*args, **kwargs)[source]¶ Gathers values along an axis specified by dim.
For a 3-D tensor, the output is:
output[i][j][k] = x[index[i][j][k]][j][k] # if dim == 0 output[i][j][k] = x[i][index[i][j][k]][k] # if dim == 1 output[i][j][k] = x[i][j][index[i][j][k]] # if dim == 2
If x is an n-D tensor with shape \((z_0, z_1, ..., z_i, ..., z_{n-1})\) and dim = i, the index must be an n-D tensor with shape \((z_0, z_1, ..., y, ..., z_{n-1})\) where y>=1 and the output will have the same shape as index.
- Inputs:
x (Tensor) - The source tensor.
dim (int) - The axis along which to index. It must be int32 or int64. Only constant value is allowed.
index (Tensor) - The indices of elements to gather. It can be one of the following data types: int32, int64. The value range of each index element is [-x_rank[dim], x_rank[dim]).
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Raises
TypeError – If dtype of dim or index is neither int32 nor int64.
ValueError – If length of shape of x is not equal to length of shape of index.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([[1, 2], [3, 4]]), mindspore.int32) >>> index = Tensor(np.array([[0, 0], [1, 0]]), mindspore.int32) >>> dim = 1 >>> output = ops.GatherD()(x, dim, index) >>> print(output) [[1 1] [4 3]]
-
class
tinyms.primitives.GatherNd(*args, **kwargs)[source]¶ Gathers slices from a tensor by indices.
Using given indices to gather slices from a tensor with a specified shape.
indices is an K-dimensional integer tensor. Supposes it as a (K-1)-dimensional tensor and each element of it defines a slice of input_x:
\[output[(i_0, ..., i_{K-2})] = input\_x[indices[(i_0, ..., i_{K-2})]]\]The last dimension of indices can not more than the rank of input_x: \(indices.shape[-1] <= input\_x.rank\).
- Inputs:
input_x (Tensor) - The target tensor to gather values.
indices (Tensor) - The index tensor, with int data type.
- Outputs:
Tensor, has the same type as input_x and the shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Raises
ValueError – If length of shape of input_x is less than the last dimension of indices.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]), mindspore.float32) >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> op = ops.GatherNd() >>> output = op(input_x, indices) >>> print(output) [-0.1 0.5]
-
class
tinyms.primitives.GatherV2(**kwargs)[source]¶ Same as operator Gather. GatherV2 will be deprecated in the future. Please use Gather instead.
-
class
tinyms.primitives.GeLU(*args, **kwargs)[source]¶ Gaussian Error Linear Units activation function.
GeLU is described in the paper Gaussian Error Linear Units (GELUs). And also please refer to BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.
GeLU is defined as follows:
\[\text{output} = 0.5 * x * (1 + erf(x / \sqrt{2})),\]where \(erf\) is the “Gauss error function” .
- Inputs:
input_x (Tensor) - Input to compute the GeLU with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as input.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> tensor = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> gelu = ops.GeLU() >>> result = gelu(tensor) >>> print(result) [0.841192 1.9545976 2.9963627]
-
class
tinyms.primitives.GeSwitch(*args, **kwargs)[source]¶ Adds control switch to data.
Switch data flows into false or true branch depending on the condition. If the condition is true, the true branch will be activated, or vise verse.
- Inputs:
data (Union[Tensor, Number]) - The data to be used for switch control.
pred (Tensor) - It must be a scalar whose type is bool and shape is (), It is used as condition for switch control.
- Outputs:
tuple. Output is tuple(false_output, true_output). The Elements in the tuple has the same shape of input data. The false_output connects with the false_branch and the true_output connects with the true_branch.
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.square = ops.Square() ... self.add = ops.Add() ... self.value = Tensor(np.full((1), 3), mindspore.float32) ... self.switch = ops.GeSwitch() ... self.merge = ops.Merge() ... self.less = ops.Less() ... ... def construct(self, x, y): ... cond = self.less(x, y) ... st1, sf1 = self.switch(x, cond) ... st2, sf2 = self.switch(y, cond) ... add_ret = self.add(st1, st2) ... st3, sf3 = self.switch(self.value, cond) ... sq_ret = self.square(sf3) ... ret = self.merge((add_ret, sq_ret)) ... return ret[0] ... >>> x = Tensor(10.0, dtype=mindspore.float32) >>> y = Tensor(5.0, dtype=mindspore.float32) >>> net = Net() >>> output = net(x, y) >>> print(output)
-
class
tinyms.primitives.Gelu(**kwargs)[source]¶ Same as operator GeLU. Gelu will be deprecated in the future. Please use GeLU instead.
-
class
tinyms.primitives.GetCenterOfGeometry(*args, **kwargs)[source]¶ Get Center Of Geometry.
- Supported Platforms:
GPU
-
class
tinyms.primitives.GetNext(*args, **kwargs)[source]¶ Returns the next element in the dataset queue.
Note
The GetNext operation needs to be associated with network and it also depends on the init_dataset interface, it can’t be used directly as a single operation. For details, please refer to connect_network_with_dataset source code.
- Parameters
- Inputs:
No inputs.
- Outputs:
tuple[Tensor], the output of Dataset. The shape is described in shapes and the type is described in types.
- Supported Platforms:
AscendGPU
Examples
>>> train_dataset = create_custom_dataset() >>> dataset_helper = mindspore.DatasetHelper(train_dataset, dataset_sink_mode=True) >>> dataset = dataset_helper.iter.dataset >>> dataset_types, dataset_shapes = dataset_helper.types_shapes() >>> queue_name = dataset.__transfer_dataset__.queue_name >>> get_next = P.GetNext(dataset_types, dataset_shapes, len(dataset_types), queue_name) >>> data, label = get_next() >>> relu = P.ReLU() >>> result = relu(data).asnumpy() >>> print(result.shape) (32, 1, 32, 32)
-
class
tinyms.primitives.Greater(*args, **kwargs)[source]¶ Computes the boolean value of \(x > y\) element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is bool.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 1, 4]), mindspore.int32) >>> greater = ops.Greater() >>> output = greater(input_x, input_y) >>> print(output) [False True False]
-
class
tinyms.primitives.GreaterEqual(*args, **kwargs)[source]¶ Computes the boolean value of \(x >= y\) element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is bool.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 1, 4]), mindspore.int32) >>> greater_equal = ops.GreaterEqual() >>> output = greater_equal(input_x, input_y) >>> print(output) [ True True False]
-
class
tinyms.primitives.HSigmoid(*args, **kwargs)[source]¶ Hard sigmoid activation function.
Applies hard sigmoid activation element-wise. The input is a Tensor with any valid shape.
Hard sigmoid is defined as:
\[\text{hsigmoid}(x_{i}) = max(0, min(1, \frac{x_{i} + 3}{6})),\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Inputs:
input_data (Tensor) - The input of HSigmoid, data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
- Supported Platforms:
GPUCPU
Examples
>>> hsigmoid = ops.HSigmoid() >>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> result = hsigmoid(input_x) >>> print(result) [0.3333 0.1666 0.5 0.833 0.6665]
-
class
tinyms.primitives.HSwish(*args, **kwargs)[source]¶ Hard swish activation function.
Applies hswish-type activation element-wise. The input is a Tensor with any valid shape.
Hard swish is defined as:
\[\text{hswish}(x_{i}) = x_{i} * \frac{ReLU6(x_{i} + 3)}{6},\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Inputs:
input_data (Tensor) - The input of HSwish, data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
- Supported Platforms:
GPUCPU
Examples
>>> hswish = ops.HSwish() >>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> result = hswish(input_x) >>> print(result) [-0.3333 -0.3333 0 1.666 0.6665]
-
class
tinyms.primitives.HistogramFixedWidth(*args, **kwargs)[source]¶ Returns a rank 1 histogram counting the number of entries in values that fall into every bin. The bins are equal width and determined by the arguments range and nbins.
- Parameters
- Inputs:
x (Tensor) - Numeric Tensor. Must be one of the following types: int32, float32, float16.
range (Tensor) - Must has the same data type as x, and the shape is [2]. x <= range[0] will be mapped to hist[0], x >= range[1] will be mapped to hist[-1].
- Outputs:
Tensor, the type is int32.
- Raises
TypeError – If dtype is not a str or nbins is not an int.
ValueError – If nbins is less than 1.
ValueError – If dtype is neither ‘int32’ nor ‘int64’.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor([-1.0, 0.0, 1.5, 2.0, 5.0, 15], mindspore.float16) >>> range = Tensor([0.0, 5.0], mindspore.float16) >>> hist = ops.HistogramFixedWidth(5) >>> output = hist(x, range) >>> print(output) [2 1 1 0 2]
-
class
tinyms.primitives.HistogramSummary(*args, **kwargs)[source]¶ Outputs the tensor to protocol buffer through histogram summary operator.
- Inputs:
name (str) - The name of the input variable.
value (Tensor) - The value of tensor, and the rank of tensor must be greater than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.nn as nn >>> import mindspore.ops as ops >>> >>> >>> class SummaryDemo(nn.Cell): ... def __init__(self,): ... super(SummaryDemo, self).__init__() ... self.summary = ops.HistogramSummary() ... self.add = ops.Add() ... ... def construct(self, x, y): ... x = self.add(x, y) ... name = "x" ... self.summary(name, x) ... return x ...
-
class
tinyms.primitives.HookBackward(hook_fn, cell_id='')[source]¶ This operation is used as a tag to hook gradient in intermediate variables. Note that this function is only supported in Pynative Mode.
Note
The hook function must be defined like hook_fn(grad) -> Tensor or None, where grad is the gradient passed to the primitive and gradient may be modified and passed to next primitive. The difference between a hook function and callback of InsertGradientOf is that a hook function is executed in the python environment while callback will be parsed and added to the graph.
- Parameters
hook_fn (Function) – Python function. hook function.
- Inputs:
inputs (Tensor) - The variable to hook.
Examples
>>> def hook_fn(grad_out): ... print(grad_out) ... >>> grad_all = GradOperation(get_all=True) >>> hook = ops.HookBackward(hook_fn) >>> def hook_test(x, y): ... z = x * y ... z = hook(z) ... z = z * y ... return z ... >>> def backward(x, y): ... return grad_all(hook_test)(x, y) ... >>> output = backward(1, 2) >>> print(output)
-
class
tinyms.primitives.IOU(*args, **kwargs)[source]¶ Calculates intersection over union for boxes.
Computes the intersection over union (IOU) or the intersection over foreground (IOF) based on the ground-truth and predicted regions.
\[ \begin{align}\begin{aligned}\text{IOU} = \frac{\text{Area of Overlap}}{\text{Area of Union}}\\\text{IOF} = \frac{\text{Area of Overlap}}{\text{Area of Ground Truth}}\end{aligned}\end{align} \]- Parameters
mode (string) – The mode is used to specify the calculation method, now supporting ‘iou’ (intersection over union) or ‘iof’ (intersection over foreground) mode. Default: ‘iou’.
- Inputs:
anchor_boxes (Tensor) - Anchor boxes, tensor of shape (N, 4). “N” indicates the number of anchor boxes, and the value “4” refers to “x0”, “y0”, “x1”, and “y1”. Data type must be float16 or float32.
gt_boxes (Tensor) - Ground truth boxes, tensor of shape (M, 4). “M” indicates the number of ground truth boxes, and the value “4” refers to “x0”, “y0”, “x1”, and “y1”. Data type must be float16 or float32.
- Outputs:
Tensor, the ‘iou’ values, tensor of shape (M, N), with the same data type as anchor_boxes.
- Raises
KeyError – When mode is not ‘iou’ or ‘iof’.
- Supported Platforms:
AscendGPU
Examples
>>> iou = ops.IOU() >>> anchor_boxes = Tensor(np.random.randint(1.0, 5.0, [3, 4]), mindspore.float16) >>> gt_boxes = Tensor(np.random.randint(1.0, 5.0, [3, 4]), mindspore.float16) >>> output = iou(anchor_boxes, gt_boxes) >>> print(output.shape) (3, 3)
-
class
tinyms.primitives.Identity(*args, **kwargs)[source]¶ Returns a Tensor with the same shape and contents as input.
- Inputs:
x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, the shape of tensor is the same as input_x, \((x_1, x_2, ..., x_R)\).
- Raises
TypeError – If x is not a Tensor.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([1, 2, 3, 4]), mindspore.int64) >>> output = ops.Identity()(x) >>> print(output) [1 2 3 4]
-
class
tinyms.primitives.ImageSummary(*args, **kwargs)[source]¶ Outputs the image tensor to protocol buffer through image summary operator.
- Inputs:
name (str) - The name of the input variable, it must not be an empty string.
value (Tensor) - The value of image, the rank of tensor must be 4.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.nn as nn >>> import mindspore.ops as ops >>> >>> >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.summary = ops.ImageSummary() ... ... def construct(self, x): ... name = "image" ... out = self.summary(name, x) ... return out ...
-
class
tinyms.primitives.InTopK(*args, **kwargs)[source]¶ Determines whether the targets are in the top k predictions.
- Parameters
k (int) – Specifies the number of top elements to be used for computing precision.
- Inputs:
x1 (Tensor) - A 2D Tensor defines the predictions of a batch of samples with float16 or float32 data type.
x2 (Tensor) - A 1D Tensor defines the labels of a batch of samples with int32 data type. The size of x2 must be equal to x1’s first dimension. The values of x2 can not be negative and must be equal to or less than index of x1’s second dimension.
- Outputs:
Tensor has 1 dimension of type bool and the same shape with x2. For labeling sample i in x2, if the label in the first k predictions for sample i is in x1, then the value is True, otherwise False.
- Raises
- Supported Platforms:
Ascend
Examples
>>> x1 = Tensor(np.array([[1, 8, 5, 2, 7], [4, 9, 1, 3, 5]]), mindspore.float32) >>> x2 = Tensor(np.array([1, 3]), mindspore.int32) >>> in_top_k = ops.InTopK(3) >>> output = in_top_k(x1, x2) >>> print(output) [ True False]
-
class
tinyms.primitives.IndexAdd(*args, **kwargs)[source]¶ Adds tensor y to specified axis and indices of tensor x. The axis should be in the range from 0 to len(x.dim) - 1, and indices should be in the range from 0 to the size of x at the axis dimension.
- Parameters
axis (int) – The dimension along which to index.
- Inputs:
input_x (Parameter) - The input tensor to add to, with data type float64, float32, float16, int32, int16, int8, uint8.
indices (Tensor) - The index of input_x on the axis th dimension to add to, with data type int32. The indices must be 1D with the same size as the size of the axis th dimension of input_y. The values of indices should be in the range of 0 to the size of the axis th dimension of input_x.
input_y (Tensor) - The input tensor with the value to add. Must have same data type as input_x. The shape must be the same as input_x except the axis th dimension.
- Outputs:
Tensor, has the same shape and dtype as input_x.
- Raises
TypeError – If dtype of input_x is not one of: float64, float32, float16, int32, int16, int8, uint8.
TypeError – If neither indices nor input_y is a Tensor.
TypeError – If shape of input_y is not same as the input_x.
ValueError – If axis is out of input_x rank’s range.
ValueError – If input_x rank is not the same as input_y rank.
ValueError – If size of indices is not equal to dimension of y[axis].
ValueError – If input_y’s shape is not the same as input_x except the axis th dimension.
- Supported Platforms:
GPU
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.index_add = ops.IndexAdd(axis=1) ... self.input_x = Parameter(Tensor(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), mindspore.float32)) ... self.indices = Tensor(np.array([0, 2]), mindspore.int32) ... ... def construct(self, input_y): ... return self.index_add(self.input_x, self.indices, input_y) ... >>> input_y = Tensor(np.array([[0.5, 1.0], [1.0, 1.5], [2.0, 2.5]]), mindspore.float32) >>> net = Net() >>> output = net(input_y) >>> print(output) [[ 1.5 2. 4. ] [ 5. 5. 7.5] [ 9. 8. 11.5]]
-
class
tinyms.primitives.InplaceAdd(*args, **kwargs)[source]¶ Adds v into specified rows of x. Computes y = x; y[i,] += v.
- Parameters
indices (Union[int, tuple]) – Indices into the left-most dimension of x, and determines which rows of x to add with v. It is an integer or a tuple, whose value is in [0, the first dimension size of x).
- Inputs:
input_x (Tensor) - The first input is a tensor whose data type is float16, float32 or int32.
input_v (Tensor) - The second input is a tensor that has the same dimension sizes as x except the first dimension, which must be the same as indices’s size. It has the same data type with input_x.
- Outputs:
Tensor, has the same shape and dtype as input_x.
- Raises
TypeError – If indices is neither int nor tuple.
TypeError – If indices is a tuple whose elements are not all int.
ValueError – If length of shape of input_x is not equal to length of shape of input_v.
- Supported Platforms:
Ascend
Examples
>>> indices = (0, 1) >>> input_x = Tensor(np.array([[1, 2], [3, 4], [5, 6]]), mindspore.float32) >>> input_v = Tensor(np.array([[0.5, 1.0], [1.0, 1.5]]), mindspore.float32) >>> inplaceAdd = ops.InplaceAdd(indices) >>> output = inplaceAdd(input_x, input_v) >>> print(output) [[1.5 3. ] [4. 5.5] [5. 6. ]]
-
class
tinyms.primitives.InplaceSub(*args, **kwargs)[source]¶ Subtracts v into specified rows of x. Computes y = x; y[i, :] -= v.
- Parameters
indices (Union[int, tuple]) – Indices into the left-most dimension of x, and determines which rows of x to subtract with v. It is a int or tuple, whose value is in [0, the first dimension size of x).
- Inputs:
input_x (Tensor) - The first input is a tensor whose data type is float16, float32 or int32.
input_v (Tensor) - The second input is a tensor who has the same dimension sizes as x except the first dimension, which must be the same as indices’s size. It has the same data type with input_x.
- Outputs:
Tensor, has the same shape and dtype as input_x.
- Raises
TypeError – If indices is neither int nor tuple.
TypeError – If indices is a tuple whose elements are not all int.
ValueError – If length of shape of input_x is not equal to length of shape of input_v.
- Supported Platforms:
Ascend
Examples
>>> indices = (0, 1) >>> input_x = Tensor(np.array([[1, 2], [3, 4], [5, 6]]), mindspore.float32) >>> input_v = Tensor(np.array([[0.5, 1.0], [1.0, 1.5]]), mindspore.float32) >>> inplaceSub = ops.InplaceSub(indices) >>> output = inplaceSub(input_x, input_v) >>> print(output) [[0.5 1. ] [2. 2.5] [5. 6. ]]
-
class
tinyms.primitives.InplaceUpdate(*args, **kwargs)[source]¶ Updates specified rows with values in v.
- Parameters
indices (Union[int, tuple]) – Indices into the left-most dimension of x, and determines which rows of x to update with v. It is a int or tuple, whose value is in [0, the first dimension size of x).
- Inputs:
x (Tensor) - A tensor which to be inplace updated. It can be one of the following data types: float32, float16 and int32.
v (Tensor) - A tensor with the same type as x and the same dimension size as x except the first dimension, which must be the same as the size of indices.
- Outputs:
Tensor, with the same type and shape as the input x.
- Raises
- Supported Platforms:
Ascend
Examples
>>> indices = (0, 1) >>> x = Tensor(np.array([[1, 2], [3, 4], [5, 6]]), mindspore.float32) >>> v = Tensor(np.array([[0.5, 1.0], [1.0, 1.5]]), mindspore.float32) >>> inplace_update = ops.InplaceUpdate(indices) >>> output = inplace_update(x, v) >>> print(output) [[0.5 1. ] [1. 1.5] [5. 6. ]]
-
class
tinyms.primitives.InsertGradientOf(*args, **kwargs)[source]¶ Attaches callback to the graph node that will be invoked on the node’s gradient.
- Parameters
f (Function) – MindSpore’s Function. Callback function.
- Inputs:
input_x (Any) - The graph node to attach to.
- Outputs:
Tensor, returns input_x directly. InsertGradientOf does not affect the forward result.
- Raises
TypeError – If f is not a function of mindspore.
- Supported Platforms:
AscendGPUCPU
Examples
>>> def clip_gradient(dx): ... ret = dx ... if ret > 1.0: ... ret = 1.0 ... ... if ret < 0.2: ... ret = 0.2 ... ... return ret ... >>> clip = ops.InsertGradientOf(clip_gradient) >>> grad_all = ops.GradOperation(get_all=True) >>> def InsertGradientOfClipDemo(): ... def clip_test(x, y): ... x = clip(x) ... y = clip(y) ... c = x * y ... return c ... ... @ms_function ... def f(x, y): ... return clip_test(x, y) ... ... def fd(x, y): ... return grad_all(clip_test)(x, y) ... ... print("forward: ", f(1.1, 0.1)) ... print("clip_gradient:", fd(1.1, 0.1)) ...
-
class
tinyms.primitives.Inv(*args, **kwargs)[source]¶ Computes Inv(Reciprocal) of input tensor element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). Must be one of the following types: float16, float32, int32.
- Outputs:
Tensor, has the same shape and data type as input_x.
- Raises
TypeError – If dtype of input_x is not one of float16, float32, int32.
- Supported Platforms:
Ascend
Examples
>>> inv = ops.Inv() >>> input_x = Tensor(np.array([0.25, 0.4, 0.31, 0.52]), mindspore.float32) >>> output = inv(input_x) >>> print(output) [4. 2.5 3.2258065 1.923077 ]
-
class
tinyms.primitives.Invert(*args, **kwargs)[source]¶ Flips all bits of input tensor element-wise.
- Inputs:
input_x (Tensor[int16], Tensor[uint16]) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If dtype of input_x is neither int16 nor uint16.
- Supported Platforms:
Ascend
Examples
>>> invert = ops.Invert() >>> input_x = Tensor(np.array([25, 4, 13, 9]), mindspore.int16) >>> output = invert(input_x) >>> print(output) [-26 -5 -14 -10]
-
class
tinyms.primitives.InvertPermutation(*args, **kwargs)[source]¶ Computes the inverse of an index permutation.
Given a tuple input, this operation inserts a dimension of 1 at the dimension This operation calculates the inverse of the index replacement. It requires a 1-dimensional tuple x, which represents the array starting at zero, and swaps each value with its index position. In other words, for the output tuple y and the input tuple x, this operation calculates the following: \(y[x[i]] = i, \quad i \in [0, 1, \ldots, \text{len}(x)-1]\).
Note
These values must include 0. There must be no duplicate values and the values can not be negative.
- Inputs:
input_x (Union(tuple[int], list[int]) - The input is constructed by multiple integers, i.e., \((y_1, y_2, ..., y_S)\) representing the indices. The values must include 0. There can be no duplicate values or negative values. Only constant value is allowed. The maximum value must be equal to length of input_x.
- Outputs:
tuple[int]. It has the same length as the input.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> invert = ops.InvertPermutation() >>> input_data = (3, 4, 0, 2, 1) >>> output = invert(input_data) >>> print(output) (2, 4, 3, 0, 1)
-
class
tinyms.primitives.IsFinite(*args, **kwargs)[source]¶ Determines which elements are finite for each position.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape of input, and the dtype is bool.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> is_finite = ops.IsFinite() >>> input_x = Tensor(np.array([np.log(-1), 1, np.log(0)]), mindspore.float32) >>> output = is_finite(input_x) >>> print(output) [False True False]
-
class
tinyms.primitives.IsInf(*args, **kwargs)[source]¶ Determines which elements are inf or -inf for each position
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape of input, and the dtype is bool.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
GPU
Examples
>>> is_inf = ops.IsInf() >>> input_x = Tensor(np.array([np.log(-1), 1, np.log(0)]), mindspore.float32) >>> output = is_inf(input_x) >>> print(output) [False False True]
-
class
tinyms.primitives.IsInstance(*args, **kwargs)[source]¶ Checks whether an object is an instance of a target type.
- Inputs:
inst (Any Object) - The instance to be checked. Only constant value is allowed.
type_ (mindspore.dtype) - The target type. Only constant value is allowed.
- Outputs:
bool, the check result.
- Raises
TypeError – If type_ is not a Type.
- Supported Platforms:
AscendGPUCPU
Examples
>>> a = 1 >>> output = ops.IsInstance()(a, mindspore.int32) >>> print(output) False
-
class
tinyms.primitives.IsNan(*args, **kwargs)[source]¶ Determines which elements are NaN for each position.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape of input, and the dtype is bool.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
GPU
Examples
>>> is_nan = ops.IsNan() >>> input_x = Tensor(np.array([np.log(-1), 1, np.log(0)]), mindspore.float32) >>> output = is_nan(input_x) >>> print(output) [True False False]
-
class
tinyms.primitives.IsSubClass(*args, **kwargs)[source]¶ Checks whether this type is a sub-class of another type.
- Inputs:
sub_type (mindspore.dtype) - The type to be checked. Only constant value is allowed.
type_ (mindspore.dtype) - The target type. Only constant value is allowed.
- Outputs:
bool, the check result.
- Raises
TypeError – If sub_type or type_ is not a Type.
- Supported Platforms:
AscendGPUCPU
Examples
>>> output = ops.IsSubClass()(mindspore.int32, mindspore.intc) >>> print(output) True
-
class
tinyms.primitives.KLDivLoss(*args, **kwargs)[source]¶ Computes the Kullback-Leibler divergence between the target and the output.
The updating formulas of KLDivLoss algorithm are as follows,
\[L = \{l_1,\dots,l_N\}^\top, \quad l_n = y_n \cdot (\log y_n - x_n)\]Then,
\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]where \(x\) represents input. \(y\) represents label. \(\ell(x, y)\) represents output.
- Parameters
reduction (str) – Specifies the reduction to be applied to the output. Its value must be one of ‘none’, ‘mean’, ‘sum’. Default: ‘mean’.
- Inputs:
input_x (Tensor) - The input Tensor. The data type must be float32.
input_y (Tensor) - The label Tensor which has the same shape as input_x. The data type must be float32.
- Outputs:
Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as input_x. Otherwise it is a scalar.
- Raises
- Supported Platforms:
GPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.kldiv_loss = ops.KLDivLoss() ... def construct(self, x, y): ... result = self.kldiv_loss(x, y) ... return result ... >>> net = Net() >>> input_x = Tensor(np.array([0.2, 0.7, 0.1]), mindspore.float32) >>> input_y = Tensor(np.array([0., 1., 0.]), mindspore.float32) >>> output = net(input_x, input_y) >>> print(output) -0.23333333
-
class
tinyms.primitives.L2Loss(*args, **kwargs)[source]¶ Calculates half of the L2 norm of a tensor without using the sqrt.
Set input_x as x and output as loss.
\[loss = sum(x ** 2) / 2\]- Inputs:
input_x (Tensor) - A input Tensor. Data type must be float16 or float32.
- Outputs:
Tensor, has the same dtype as input_x. The output tensor is the value of loss which is a scalar tensor.
- Raises
- Supported Platforms:
AscendGPU- Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float16) >>> l2_loss = ops.L2Loss() >>> output = l2_loss(input_x) >>> print(output) 7.0
-
class
tinyms.primitives.L2Normalize(*args, **kwargs)[source]¶ L2 normalization Operator.
This operator will normalize the input using the given axis. The function is shown as follows:
\[\text{output} = \frac{x}{\sqrt{\text{max}(\text{sum} (\text{input_x}^2), \epsilon)}},\]where \(\epsilon\) is epsilon.
- Parameters
- Inputs:
input_x (Tensor) - Input to compute the normalization. Data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> l2_normalize = ops.L2Normalize() >>> input_x = Tensor(np.random.randint(-256, 256, (2, 3, 4)), mindspore.float32) >>> output = l2_normalize(input_x) >>> print(output.shape) (2, 3, 4)
-
class
tinyms.primitives.LARSUpdate(*args, **kwargs)[source]¶ Conducts LARS (layer-wise adaptive rate scaling) update on the sum of squares of gradient.
- Parameters
- Inputs:
weight (Tensor) - A tensor, representing the weight.
gradient (Tensor) - The gradient of weight, which has the same shape and dtype with weight.
norm_weight (Tensor) - A scalar tensor, representing the sum of squares of weight.
norm_gradient (Tensor) - A scalar tensor, representing the sum of squares of gradient.
weight_decay (Union[Number, Tensor]) - Weight decay. It must be a scalar tensor or number.
learning_rate (Union[Number, Tensor]) - Learning rate. It must be a scalar tensor or number.
- Outputs:
Tensor, represents the new gradient.
- Raises
TypeError – If neither epsilon nor hyperpara is a float.
TypeError – If use_clip is a bool.
TypeError – If weight, gradient, norm_weight or norm_gradient is not a Tensor.
TypeError – If weight_decay or learning_rate is neither a Number nor a Tensor.
TypeError – If shape of gradient is not same as weight.
- Supported Platforms:
Ascend
Examples
>>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> import mindspore.nn as nn >>> import numpy as np >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.lars = ops.LARSUpdate() ... self.reduce = ops.ReduceSum() ... self.square = ops.Square() ... def construct(self, weight, gradient): ... w_square_sum = self.reduce(self.square(weight)) ... grad_square_sum = self.reduce(self.square(gradient)) ... grad_t = self.lars(weight, gradient, w_square_sum, grad_square_sum, 0.0, 1.0) ... return grad_t ... >>> np.random.seed(0) >>> weight = np.random.random(size=(2, 3)).astype(np.float32) >>> gradient = np.random.random(size=(2, 3)).astype(np.float32) >>> net = Net() >>> output = net(Tensor(weight), Tensor(gradient)) >>> print(output) [[0.00036534 0.00074454 0.00080456] [0.00032014 0.00066101 0.00044157]]
-
class
tinyms.primitives.LJEnergy(*args, **kwargs)[source]¶ Calculate the Van der Waals interaction energy described by Lennard-Jones potential for each atom. Assume the number of atoms is N, and the number of Lennard-Jones types for all atoms is P, which means there will be Q = P*(P+1)/2 types of possible Lennard-Jones interactions for all kinds of atom pairs.
\[dr = (x_a-x_b, y_a-y_b, z_a-z_b)\]\[E = A/|dr|^{12} - B/|dr|^{6}\]- Agrs:
atom_numbers(int32): the number of atoms, N. cutoff_square(float32): the square value of cutoff.
- Inputs:
uint_crd (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJtype (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge carried by each atom.
scaler (Tensor, float32) - [3,], the scale factor between real space coordinate and its unsigned int value.
nl_numbers - (Tensor, int32) - [N,], the each atom.
nl_serial - (Tensor, int32) - [N, 800], the neighbor list of each atom, the max number is 800.
d_LJ_A (Tensor, float32) - [Q,], the Lennard-Jones A coefficient of each kind of atom pair. Q is the number of atom pair.
d_LJ_B (Tensor, float32) - [Q,], the Lennard-Jones B coefficient of each kind of atom pair. Q is the number of atom pair.
- Outputs:
d_LJ_energy_atom (Tensor, float32) - [N,], the Lennard-Jones potential energy of each atom.
d_LJ_energy_sum (float32), the sum of Lennard-Jones potential energy of each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.LJForce(*args, **kwargs)[source]¶ Calculate the Van der Waals interaction force described by Lennard-Jones potential energy for each atom.
\[dr = (x_a-x_b, y_a-y_b, z_a-z_b)\]\[F = (-12*A/|dr|^{14} + 6*B/|dr|^{8}) * dr\]- Agrs:
atom_numbers(int32): the number of atoms, N. cutoff_square(float32): the square value of cutoff.
- Inputs:
uint_crd (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJtype (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge carried by each atom.
scaler (Tensor, float32) - [3,], the scale factor between real space coordinate and its unsigned int value.
nl_numbers - (Tensor, int32) - [N,], the each atom.
nl_serial - (Tensor, int32) - [N, 800], the neighbor list of each atom, the max number is 800.
d_LJ_A (Tensor, float32) - [Q,], the Lennard-Jones A coefficient of each kind of atom pair. Q is the number of atom pair.
d_LJ_B (Tensor, float32) - [Q,], the Lennard-Jones B coefficient of each kind of atom pair. Q is the number of atom pair.
- outputs:
frc (Tensor, float32) - [N, 3], the force felt by each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.LJForceWithPMEDirectForce(*args, **kwargs)[source]¶ Calculate the Lennard-Jones force and PME direct force together.
The calculation formula of Lennard-Jones part is the same as operator LJForce(), and the PME direct part is within PME method.
- Agrs:
atom_numbers(int32): the number of atoms, N. cutoff_square(float32): the square value of cutoff. pme_beta(float32): PME beta parameter, same as operator PMEReciprocalForce().
- Inputs:
uint_crd (Tensor, uint32) - [N, 3], the unsigned int coordinate value of each atom.
LJtype (Tensor, int32) - [N,], the Lennard-Jones type of each atom.
charge (Tensor, float32) - [N,], the charge carried by each atom.
scaler (Tensor, float32) - [3,], the scale factor between real space coordinate and its unsigned int value.
nl_numbers - (Tensor, int32) - [N,], the each atom.
nl_serial - (Tensor, int32) - [N, 800], the neighbor list of each atom, the max number is 800.
d_LJ_A (Tensor, float32) - [Q,], the Lennard-Jones A coefficient of each kind of atom pair. Q is the number of atom pair.
d_LJ_B (Tensor, float32) - [Q,], the Lennard-Jones B coefficient of each kind of atom pair. Q is the number of atom pair.
- Outputs:
frc (Tensor, float32), [N, 3], the force felt by each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.LRN(*args, **kwargs)[source]¶ Local Response Normalization.
\[b_{c} = a_{c}\left(k + \frac{\alpha}{n} \sum_{c'=\max(0, c-n/2)}^{\min(N-1,c+n/2)}a_{c'}^2\right)^{-\beta}\]- Parameters
depth_radius (int) – Half-width of the 1-D normalization window with the shape of 0-D.
bias (float) – An offset (usually positive to avoid dividing by 0).
alpha (float) – A scale factor, usually positive.
beta (float) – An exponent.
norm_region (str) – Specifies normalization region. Options: “ACROSS_CHANNELS”. Default: “ACROSS_CHANNELS”.
- Inputs:
x (Tensor) - A 4D Tensor with float16 or float32 data type.
- Outputs:
Tensor, with the same shape and data type as the input tensor.
- Raises
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(1, 2, 2, 2), mindspore.float32) >>> lrn = ops.LRN() >>> output = lrn(x) >>> print(output.shape) (1, 2, 2, 2)
-
class
tinyms.primitives.LSTM(*args, **kwargs)[source]¶ Performs the Long Short-Term Memory (LSTM) on the input.
For detailed information, please refer to nn.LSTM.
- Parameters
input_size (int) – Number of features of input.
hidden_size (int) – Number of features of hidden layer.
num_layers (int) – Number of layers of stacked LSTM . Default: 1.
has_bias (bool) – Whether the cell has bias b_ih and b_hh. Default: True.
bidirectional (bool) – Specifies whether it is a bidirectional LSTM. Default: False.
dropout (float) – If not 0, append Dropout layer on the outputs of each LSTM layer except the last layer. Default 0. The range of dropout is [0.0, 1.0].
- Inputs:
input (Tensor) - Tensor of shape (seq_len, batch_size, input_size) or (batch_size, seq_len, input_size).
h (tuple) - Tensor of shape (num_directions * num_layers, batch_size, hidden_size).
c (tuple) - Tensor of shape (num_directions * num_layers, batch_size, hidden_size).
- Outputs:
Tuple, a tuple contains (output, h_n, c_n, reserve, state).
output (Tensor) - Tensor of shape (seq_len, batch_size, num_directions * hidden_size).
h_n (Tensor) - Tensor of shape (num_directions * num_layers, batch_size, hidden_size).
c_n (Tensor) - Tensor of shape (num_directions * num_layers, batch_size, hidden_size).
reserve (Tensor) - Tensor of shape (r, 1).
state (Tensor) - Random number generator state and its shape is (s, 1).
- Raises
TypeError – If input_size, hidden_size or num_layers is not an int.
TypeError – If has_bias or bidirectional is not a bool.
TypeError – If dropout is not a float.
ValueError – If dropout is not in range [0.0, 1.0].
- Supported Platforms:
GPUCPU
Examples
>>> input_size = 10 >>> hidden_size = 2 >>> num_layers = 1 >>> seq_len = 5 >>> batch_size = 2 >>> >>> net = P.LSTM(input_size, hidden_size, num_layers, True, False, 0.0) >>> input = Tensor(np.ones([seq_len, batch_size, input_size]).astype(np.float32)) >>> h0 = Tensor(np.ones([num_layers, batch_size, hidden_size]).astype(np.float32)) >>> c0 = Tensor(np.ones([num_layers, batch_size, hidden_size]).astype(np.float32)) >>> w = Tensor(np.ones([112, 1, 1]).astype(np.float32)) >>> output, hn, cn, _, _ = net(input, h0, c0, w) >>> print(output) [[[0.9640267 0.9640267 ] [0.9640267 0.9640267 ]] [[0.9950539 0.9950539 ] [0.9950539 0.9950539 ]] [[0.99932843 0.99932843] [0.99932843 0.99932843]] [[0.9999084 0.9999084 ] [0.9999084 0.9999084 ]] [[0.9999869 0.9999869 ] [0.9999869 0.9999869 ]]]
-
class
tinyms.primitives.LayerNorm(*args, **kwargs)[source]¶ Applies the Layer Normalization to the input tensor.
This operator will normalize the input tensor on given axis. LayerNorm is described in the paper Layer Normalization.
\[y = \frac{x - mean}{\sqrt{variance + \epsilon}} * \gamma + \beta\]where \(\gamma\) is scale, \(\beta\) is bias, \(\epsilon\) is epsilon.
- Parameters
begin_norm_axis (int) – The begin axis of the input_x to apply LayerNorm, the value must be in [-1, rank(input)). Default: 1.
begin_params_axis (int) – The begin axis of the parameter input (gamma, beta) to apply LayerNorm, the value must be in [-1, rank(input)). Default: 1.
epsilon (float) – A value added to the denominator for numerical stability. Default: 1e-7.
- Inputs:
input_x (Tensor) - Tensor of shape \((N, \ldots)\). The input of LayerNorm.
gamma (Tensor) - Tensor of shape \((P_0, \ldots, P_\text{begin_params_axis})\). The learnable parameter gamma as the scale on norm.
beta (Tensor) - Tensor of shape \((P_0, \ldots, P_\text{begin_params_axis})\). The learnable parameter beta as the scale on norm.
- Outputs:
tuple[Tensor], tuple of 3 tensors, the normalized input and the updated parameters.
output_x (Tensor) - The normalized input, has the same type and shape as the input_x. The shape is \((N, C)\).
mean (Tensor) - Tensor of shape \((C,)\).
variance (Tensor) - Tensor of shape \((C,)\).
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[1, 2, 3], [1, 2, 3]]), mindspore.float32) >>> gamma = Tensor(np.ones([3]), mindspore.float32) >>> beta = Tensor(np.ones([3]), mindspore.float32) >>> layer_norm = ops.LayerNorm() >>> output, mean, variance = layer_norm(input_x, gamma, beta) >>> print(output) [[-0.2247448 1. 2.2247448] [-0.2247448 1. 2.2247448]] >>> print(mean) [[2.] [2.]] >>> print(variance) [[0.6666667] [0.6666667]]
-
class
tinyms.primitives.Less(*args, **kwargs)[source]¶ Computes the boolean value of \(x < y\) element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 1, 4]), mindspore.int32) >>> less = ops.Less() >>> output = less(input_x, input_y) >>> print(output) [False False True]
-
class
tinyms.primitives.LessEqual(*args, **kwargs)[source]¶ Computes the boolean value of \(x <= y\) element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool , and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 1, 4]), mindspore.int32) >>> less_equal = ops.LessEqual() >>> output = less_equal(input_x, input_y) >>> print(output) [ True False True]
-
class
tinyms.primitives.LinSpace(*args, **kwargs)[source]¶ Generates values in an interval (inclusive of start and stop) and returns the corresponding interpolated array with num number of ticks.
- Inputs:
start (Tensor[float32]) - Start value of interval, With shape of 0-D.
stop (Tensor[float32]) - Last value of interval, With shape of 0-D.
num (int) - Number of ticks in the interval, inclusive of start and stop.
- Outputs:
Tensor, has the same shape as start.
- Supported Platforms:
AscendGPU
Examples
>>> linspace = P.LinSpace() >>> start = Tensor(1, mindspore.float32) >>> stop = Tensor(10, mindspore.float32) >>> num = 5 >>> output = linspace(start, stop, num) >>> print(output) [ 1. 3.25 5.5 7.75 10. ]
-
class
tinyms.primitives.Log(*args, **kwargs)[source]¶ Returns the natural logarithm of a tensor element-wise.
\[y_i = log_e(x_i)\]- Inputs:
input_x (Tensor) - The input tensor. The value must be greater than 0.
- Outputs:
Tensor, has the same shape as the input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> log = ops.Log() >>> output = log(input_x) >>> print(output) [0. 0.6931472 1.3862944]
-
class
tinyms.primitives.Log1p(*args, **kwargs)[source]¶ Returns the natural logarithm of one plus the input tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor. With float16 or float32 data type. The value must be greater than -1.
- Outputs:
Tensor, has the same shape as the input_x.
- Raises
TypeError – If dtype of input_x is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> log1p = ops.Log1p() >>> output = log1p(input_x) >>> print(output) [0.6931472 1.0986123 1.609438 ]
-
class
tinyms.primitives.LogSoftmax(*args, **kwargs)[source]¶ Log Softmax activation function.
Applies the Log Softmax function to the input tensor on the specified axis. Suppose a slice in the given aixs, \(x\) for each element \(x_i\), the Log Softmax function is shown as follows:
\[\text{output}(x_i) = \log \left(\frac{\exp(x_i)} {\sum_{j = 0}^{N-1}\exp(x_j)}\right),\]where \(N\) is the length of the Tensor.
- Parameters
axis (int) – The axis to perform the Log softmax operation. Default: -1.
- Inputs:
logits (Tensor) - The input of Log Softmax, with float16 or float32 data type.
- Outputs:
Tensor, with the same type and shape as the logits.
- Raises
TypeError – If axis is not an int.
TypeError – If dtype of logits is neither float16 nor float32.
ValueError – If axis is not in range [-len(logits), len(logits)].
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> log_softmax = ops.LogSoftmax() >>> output = log_softmax(input_x) >>> print(output) [-4.4519143 -3.4519143 -2.4519143 -1.4519144 -0.4519144]
-
class
tinyms.primitives.LogUniformCandidateSampler(*args, **kwargs)[source]¶ Generates random labels with a log-uniform distribution for sampled_candidates.
Random sampling a tensor of sampled classes from the range of integers [0, range_max).
- Parameters
num_true (int) – The number of target classes per training example. Default: 1.
num_sampled (int) – The number of classes to randomly sample. Default: 5.
unique (bool) – Determines whether sample with rejection. If unique is True, all sampled classes in a batch are unique. Default: True.
range_max (int) – The number of possible classes. When unique is True, range_max must be greater than or equal to num_sampled. Default: 5.
seed (int) – Random seed, must be non-negative.
- Inputs:
true_classes (Tensor) - The target classes. With data type of int64 and shape [batch_size, num_true].
- Outputs:
Tuple of 3 Tensors.
sampled_candidates (Tensor) - A Tensor with shape (num_sampled,) and the same type as true_classes.
true_expected_count (Tensor) - A Tensor with the same shape as true_classes and type float32.
sampled_expected_count (Tensor) - A Tensor with the same shape as sampled_candidates and type float32.
- Raises
- Supported Platforms:
Ascend
Examples
>>> sampler = ops.LogUniformCandidateSampler(2, 5, True, 5) >>> output1, output2, output3 = sampler(Tensor(np.array([[1, 7], [0, 4], [3, 3]]))) >>> print(output1, output2, output3) [3 2 0 4 1] [[0.92312991 0.49336370] [0.99248987 0.65806371] [0.73553443 0.73553443]] [0.73553443 0.82625800 0.99248987 0.65806371 0.92312991]
-
class
tinyms.primitives.LogicalAnd(*args, **kwargs)[source]¶ Computes the “logical AND” of two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one bool. When the inputs are two tensors, the shapes of them could be broadcast, and the data types of them must be bool. When the inputs are one tensor and one bool, the bool object could only be a constant, and the data type of the tensor must be bool.
- Inputs:
input_x (Union[Tensor, bool]) - The first input is a bool or a tensor whose data type is bool.
input_y (Union[Tensor, bool]) - The second input is a bool when the first input is a tensor or a tensor whose data type is bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is bool.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([True, False, True]), mindspore.bool_) >>> input_y = Tensor(np.array([True, True, False]), mindspore.bool_) >>> logical_and = ops.LogicalAnd() >>> output = logical_and(input_x, input_y) >>> print(output) [ True False False]
-
class
tinyms.primitives.LogicalNot(*args, **kwargs)[source]¶ Computes the “logical NOT” of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor whose dtype is bool.
- Outputs:
Tensor, the shape is the same as the input_x, and the dtype is bool.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([True, False, True]), mindspore.bool_) >>> logical_not = ops.LogicalNot() >>> output = logical_not(input_x) >>> print(output) [False True False]
-
class
tinyms.primitives.LogicalOr(*args, **kwargs)[source]¶ Computes the “logical OR” of two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one bool. When the inputs are two tensors, the shapes of them could be broadcast, and the data types of them must be bool. When the inputs are one tensor and one bool, the bool object could only be a constant, and the data type of the tensor must be bool.
- Inputs:
input_x (Union[Tensor, bool]) - The first input is a bool or a tensor whose data type is bool.
input_y (Union[Tensor, bool]) - The second input is a bool when the first input is a tensor or a tensor whose data type is bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Raises
TypeError – If neither input_x nor input_y is a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([True, False, True]), mindspore.bool_) >>> input_y = Tensor(np.array([True, True, False]), mindspore.bool_) >>> logical_or = ops.LogicalOr() >>> output = logical_or(input_x, input_y) >>> print(output) [ True True True]
-
class
tinyms.primitives.MDIterationLeapFrog(*args, **kwargs)[source]¶ One step of classical leap frog algorithm to solve the finite difference Hamiltonian equations of motion for certain system, using Langevin dynamics with Liu’s thermostat scheme. Assume the number of atoms is N and the target control temperature is T.
Detailed iteration formula can be found in this paper: A unified thermostat scheme for efficient configurational sampling for classical/quantum canonical ensembles via molecular dynamics. DOI: 10.1063/1.4991621.
- Parameters
float4_numbers (int32) – total length to store random numbers.
atom_numbers (int32) – the number of atoms N.
dt (float32) – time step for finite difference.
half_dt (float32) – half of time step for finite difference.
exp_gamma (float32) – parameter in Liu’s dynamic, equals exp(-gamma_ln * dt), where gamma_ln is the firction factor in Langvin dynamics.
max_velocity (float32) – the upper limit of velocity, when the veclocity overflows, scale it to the upper limit.
is_max_velocity (int32) – whether the max velocity control is open or not.
- Inputs:
mass_inverse (Tensor, float32) - [N,], the inverse value of mass of each atom.
sqrt_mass (Tensor, float32) - [N,], the inverse square root value of effect mass in Liu’s dynamics of each atom.
- Outputs:
vel (Tensor, float32) - [N, 3], the velocity of each atom.
crd (Tensor, float32) - [N, 3], the coordinate of each atom.
frc (Tensor, float32) - [N, 3], the force felt by each atom.
acc (Tensor, float32) - [N, 3], the acceleration of each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.MDIterationLeapFrogLiujian(*args, **kwargs)[source]¶ One step of classical leap frog algorithm to solve the finite difference Hamiltonian equations of motion for certain system, using Langevin dynamics with Liu’s thermostat scheme. Assume the number of atoms is N and the target control temperature is T.
Detailed iteration formula can be found in this paper: A unified thermostat scheme for efficient configurational sampling for classical/quantum canonical ensembles via molecular dynamics. DOI: 10.1063/1.4991621.
- Parameters
atom_numbers (int32) – the number of atoms N.
dt (float32) – time step for finite difference.
half_dt (float32) – half of time step for finite difference.
exp_gamma (float32) – parameter in Liu’s dynamic, equals
exp (-gamma_ln * dt) –
dynamics. –
- Inputs:
inverse_mass (Tensor, float32) - [N,], the inverse value of
mass of each atom. - sqrt_mass_inverse (Tensor, float32) - [N,], the inverse square root value of effect mass in Liu’s dynamics of each atom. - vel (Tensor, float32) - [N, 3], the velocity of each atom. - crd (Tensor, float32) - [N, 3], the coordinate of each atom. - frc (Tensor, float32) - [N, 3], the force felt by each atom. - acc (Tensor, float32) - [N, 3], the acceleration of each atom. - rand_state (Tensor, float32) - [math.ceil(atom_numbers * 3.0 / 4.0) * 16, ], random state to generate random force. - rand_frc (Tensor, float32) - [N, 3], the random forces.
- Outputs:
output (float32)
- Supported Platforms:
GPU
-
class
tinyms.primitives.MDIterationSetupRandState(*args, **kwargs)[source]¶ Convert FP32 coordinate to Uint32 coordinate.
- Parameters
atom_numbers (int32) – the number of atoms N.
seed (int32) – random seed.
- Outputs:
output (uint32) random state.
- Supported Platforms:
GPU
-
class
tinyms.primitives.MDTemperature(*args, **kwargs)[source]¶ Calculate the MD Temperature.
Calculate the temperature.
- Supported Platforms:
GPU
-
class
tinyms.primitives.MakeRefKey(*args, **kwargs)[source]¶ Makes a RefKey instance by string. RefKey stores the name of Parameter, can be passed through the functions, and used for Assign target.
- Parameters
tag (str) – Parameter name to make the RefKey.
- Inputs:
No inputs.
- Outputs:
RefKeyType, made from the Parameter name.
- Raises
TypeError – If tag is not a str.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import numpy as np >>> from mindspore import Parameter, Tensor >>> from mindspore import dtype as mstype >>> import mindspore.ops as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.y = Parameter(Tensor(np.ones([2, 3]), mstype.int32), name="y") ... self.make_ref_key = ops.MakeRefKey("y") ... ... def construct(self, x): ... key = self.make_ref_key() ... ref = ops.make_ref(key, x, self.y) ... return ref * x ... >>> x = Tensor(np.array([[1, 2, 3], [4, 5, 6]]), mindspore.int32) >>> net = Net() >>> output = net(x) >>> print(output) [[ 1 4 9] [16 25 36]]
-
class
tinyms.primitives.MatMul(*args, **kwargs)[source]¶ Multiplies matrix a and matrix b.
The rank of input tensors must equal to 2.
- Parameters
- Inputs:
input_x (Tensor) - The first tensor to be multiplied. The shape of the tensor is \((N, C)\). If transpose_a is True, its shape must be \((N, C)\) after transpose.
input_y (Tensor) - The second tensor to be multiplied. The shape of the tensor is \((C, M)\). If transpose_b is True, its shape must be \((C, M)\) after transpose.
- Outputs:
Tensor, the shape of the output tensor is \((N, M)\).
- Raises
TypeError – If transpose_a or transpose_b is not a bool.
ValueError – If the column of matrix dimensions of input_x is not equal to the row of matrix dimensions of input_y.
ValueError – If length of shape of input_x or input_y is not equal to 2.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x1 = Tensor(np.ones(shape=[1, 3]), mindspore.float32) >>> input_x2 = Tensor(np.ones(shape=[3, 4]), mindspore.float32) >>> matmul = ops.MatMul() >>> output = matmul(input_x1, input_x2) >>> print(output) [[3. 3. 3. 3.]]
-
class
tinyms.primitives.MatrixInverse(*args, **kwargs)[source]¶ Returns the inverse of the input matrix. If the matrix is irreversible, an error may be reported or an unknown result may be returned.
Note
The parameter ‘adjoint’ is only supporting False right now. Because complex number is not supported at present.
- Parameters
adjoint (bool) – An optional bool. Default: False.
- Inputs:
x (Tensor) - A matrix to be calculated. The matrix must be at least two dimensions, and the last two dimensions must be the same size. types: float32, float64.
- Outputs:
Tensor, has the same type and shape as input x.
- Raises
TypeError – If adjoint is not a bool.
TypeError – If dtype of x is neither float32 nor float64.
ValueError – If the last two dimensions of x is not same size.
ValueError – If the dimension of x is less than 2.
- Supported Platforms:
GPU
Examples
>>> x = Tensor(np.array([[[-0.710504 , -1.1207525], ... [-1.7651395 , -1.7576632]], ... [[ 0.52412605, 1.9070215], ... [ 1.3384849 , 1.4274558]]]), mindspore.float32) >>> matrix_inverse = ops.MatrixInverse(adjoint=False) >>> output = matrix_inverse(x) >>> print(output) [[[ 2.4095483 -1.536419 ] [-2.4197974 0.97401696]] [[-0.79111797 1.0569006 ] [ 0.74180895 -0.2904787 ]]]
-
class
tinyms.primitives.MaxPool(*args, **kwargs)[source]¶ Max pooling operation.
Applies a 2D max pooling over an input Tensor which can be regarded as a composition of 2D planes.
Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), MaxPool outputs regional maximum in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \max_{m=0, \ldots, h_{ker}-1} \max_{n=0, \ldots, w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the maximum value, is an int number that represents height and width are both kernel_size, or a tuple of two int numbers that represent height and width respectively. Default: 1.
strides (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) – The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
data_format (str) –
The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, with shape \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If kernel_size or strides is neither int nor tuple.
ValueError – If pad_mode is neither ‘valid’ nor ‘same’ with not case sensitive.
ValueError – If data_format is neither ‘NCHW’ nor ‘NHWC’.
ValueError – If kernel_size or strides is less than 1.
ValueError – If length of shape of input is not equal to 4.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.arange(1 * 3 * 3 * 4).reshape((1, 3, 3, 4)), mindspore.float32) >>> maxpool_op = ops.MaxPool(pad_mode="VALID", kernel_size=2, strides=1) >>> output = maxpool_op(input_tensor) >>> print(output) [[[[ 5. 6. 7.] [ 9. 10. 11.]] [[17. 18. 19.] [21. 22. 23.]] [[29. 30. 31.] [33. 34. 35.]]]]
-
class
tinyms.primitives.MaxPool3D(*args, **kwargs)[source]¶ 3D max pooling operation.
Applies a 3D max pooling over an input Tensor which can be regarded as a composition of 3D planes.
Typically the input is of shape \((N_{in}, C_{in}, D_{in}, H_{in}, W_{in})\), MaxPool outputs regional maximum in the \((D_{in}, H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (d_{ker}, h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1, s_2)\), the operation is as follows.
\[\text{output}(N_i, C_j, d, h, w) = \max_{l=0, \ldots, d_{ker}-1} \max_{m=0, \ldots, h_{ker}-1} \max_{n=0, \ldots, w_{ker}-1} \text{input}(N_i, C_j, s_0 \times d + l, s_1 \times h + m, s_2 \times w + n)\]- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the maximum value, is an int number that represents height and width are both kernel_size, or a tuple of three int numbers that represent depth, height and width respectively. Default: 1.
strides (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the depth, height and width of movement are both strides, or a tuple of three int numbers that represent depth, height and width of movement respectively. Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
data_format (str) – The optional value for data format. Currently only support ‘NCDHW’. Default: ‘NCDHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C, D_{in}, H_{in}, W_{in})\). Data type must be float16.
- Outputs:
Tensor, with shape \((N, C, D_{out}, H_{out}, W_{out})\). Has the data type with input.
- Raises
TypeError – If kernel_size or strides is neither an int not a tuple.
TypeError – If pad_mode or data_format is not a string.
ValueError – If numbers in kernel_size or strides are not positive.
ValueError – If pad_mode is not one of ‘same’, ‘valid’.
ValueError – If kernel_size or strides is a tuple whose length is not equal to 3.
ValueError – If data_format is not ‘NCDHW’.
- Supported Platforms:
Ascend
Examples
>>> input = Tensor(np.arange(1 * 2 * 2 * 2 * 3).reshape((1, 2, 2, 2, 3)), mindspore.float32) >>> max_pool3d = ops.MaxPool3D(kernel_size=2, strides=1, pad_mode="valid") >>> output = max_pool3d(input) >>> print(output) [[[[[10. 11.]]] [[[22. 23.]]]]]
-
class
tinyms.primitives.MaxPoolWithArgmax(*args, **kwargs)[source]¶ Performs max pooling on the input Tensor and returns both max values and indices.
Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), MaxPool outputs regional maximum in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \max_{m=0, \ldots, h_{ker}-1} \max_{n=0, \ldots, w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the maximum value and arg value, is an int number that represents height and width are both kernel_size, or a tuple of two int numbers that represent height and width respectively. Default: 1.
strides (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\). Data type must be float16 or float32.
- Outputs:
Tuple of 2 Tensors, representing the maxpool result and where the max values are generated.
output (Tensor) - Maxpooling result, with shape \((N, C_{out}, H_{out}, W_{out})\). It has the same data type as input.
mask (Tensor) - Max values’ index represented by the mask. Data type is int32.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> input_tensor = Tensor(np.arange(1 * 3 * 3 * 4).reshape((1, 3, 3, 4)), mindspore.float32) >>> maxpool_arg_op = ops.MaxPoolWithArgmax(pad_mode="VALID", kernel_size=2, strides=1) >>> output_tensor, argmax = maxpool_arg_op(input_tensor) >>> print(output_tensor) [[[[ 5. 6. 7.] [ 9. 10. 11.]] [[17. 18. 19.] [21. 22. 23.]] [[29. 30. 31.] [33. 34. 35.]]]]
-
class
tinyms.primitives.Maximum(*args, **kwargs)[source]¶ Computes the maximum of input tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
ValueError – If input_x and input_y are not the same shape.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 5.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([4.0, 2.0, 6.0]), mindspore.float32) >>> maximum = ops.Maximum() >>> output = maximum(input_x, input_y) >>> print(output) [4. 5. 6.]
-
class
tinyms.primitives.Merge(*args, **kwargs)[source]¶ Merges all input data to one.
One and only one of the inputs must be selected as the output
- Inputs:
inputs (Union(Tuple, List)) - The data to be merged. All tuple elements must have the same data type.
- Outputs:
tuple. Output is tuple(data, output_index). The data has the same shape of inputs element.
- Raises
TypeError – If inputs is neither Tuple nor list.
Examples
>>> merge = ops.Merge() >>> input_x = Tensor(np.linspace(0, 8, 8).reshape(2, 4), mindspore.float32) >>> input_y = Tensor(np.random.randint(-4, 4, (2, 4)), mindspore.float32) >>> result = merge((input_x, input_y))
-
class
tinyms.primitives.Meshgrid(*args, **kwargs)[source]¶ Generates coordinate matrices from given coordinate tensors.
Given N one-dimensional coordinate tensors, returns a tuple outputs of N N-D coordinate tensors for evaluating expressions on an N-D grid.
- Parameters
indexing (str) – Either ‘xy’ or ‘ij’. Default: ‘xy’. When the indexing argument is set to ‘xy’ (the default), the broadcasting instructions for the first two dimensions are swapped.
- Inputs:
input (Union[tuple]) - A Tuple of N 1-D Tensor objects. The length of input should be greater than 1
- Outputs:
Tensors, A Tuple of N N-D Tensor objects.
- Raises
TypeError – If indexing is not a str or input is not a tuple.
ValueError – If indexing is neither ‘xy’ nor ‘ij’.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32)) >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32)) >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32)) >>> inputs = (x, y, z) >>> meshgrid = ops.Meshgrid(indexing="xy") >>> output = meshgrid(inputs) >>> print(output) (Tensor(shape=[3, 4, 5], dtype=Int32, value= [[[1, 1, 1, 1, 1], [2, 2, 2, 2, 2], [3, 3, 3, 3, 3], [4, 4, 4, 4, 4]], [[1, 1, 1, 1, 1], [2, 2, 2, 2, 2], [3, 3, 3, 3, 3], [4, 4, 4, 4, 4]], [[1, 1, 1, 1, 1], [2, 2, 2, 2, 2], [3, 3, 3, 3, 3], [4, 4, 4, 4, 4]]]), Tensor(shape=[3, 4, 5], dtype=Int32, value= [[[5, 5, 5, 5, 5], [5, 5, 5, 5, 5], [5, 5, 5, 5, 5], [5, 5, 5, 5, 5]], [[6, 6, 6, 6, 6], [6, 6, 6, 6, 6], [6, 6, 6, 6, 6], [6, 6, 6, 6, 6]], [[7, 7, 7, 7, 7], [7, 7, 7, 7, 7], [7, 7, 7, 7, 7], [7, 7, 7, 7, 7]]]), Tensor(shape=[3, 4, 5], dtype=Int32, value= [[[8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2]], [[8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2]], [[8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2], [8, 9, 0, 1, 2]]]))
-
class
tinyms.primitives.Minimum(*args, **kwargs)[source]¶ Computes the minimum of input tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
ValueError – If input_x and input_y are not the same shape.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 5.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([4.0, 2.0, 6.0]), mindspore.float32) >>> minimum = ops.Minimum() >>> output = minimum(input_x, input_y) >>> print(output) [1. 2. 3.]
-
class
tinyms.primitives.MirrorPad(*args, **kwargs)[source]¶ Pads the input tensor according to the paddings and mode.
- Parameters
mode (str) – Specifies the padding mode. The optional values are “REFLECT” and “SYMMETRIC”. Default: “REFLECT”.
- Inputs:
input_x (Tensor) - The input tensor.
paddings (Tensor) - The paddings tensor. The value of paddings is a matrix(list), and its shape is (N, 2). N is the rank of input data. All elements of paddings are int type. For the input in the D th dimension, paddings[D, 0] indicates how many sizes to be extended ahead of the input tensor in the D th dimension, and paddings[D, 1] indicates how many sizes to be extended behind the input tensor in the D th dimension.
- Outputs:
Tensor, the tensor after padding.
If mode is “REFLECT”, it uses a way of symmetrical copying through the axis of symmetry to fill in. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[6,5,4,5,6,5,4], [3,2,1,2,3,2,1], [6,5,4,5,6,5,4], [9,8,7,8,9,8,7], [6,5,4,5,6,5,4]].
If mode is “SYMMETRIC”, the filling method is similar to the “REFLECT”. It is also copied according to the symmetry axis, except that it includes the symmetry axis. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[2,1,1,2,3,3,2], [2,1,1,2,3,3,2], [5,4,4,5,6,6,5], [8,7,7,8,9,9,8], [8,7,7,8,9,9,8]].
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> import mindspore.nn as nn >>> import numpy as np >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.pad = ops.MirrorPad(mode="REFLECT") ... def construct(self, x, paddings): ... return self.pad(x, paddings) ... >>> x = np.random.random(size=(2, 3)).astype(np.float32) >>> paddings = Tensor([[1, 1], [2, 2]]) >>> pad = Net() >>> output = pad(Tensor(x), paddings) >>> print(output.shape) (4, 7)
-
class
tinyms.primitives.Mish(*args, **kwargs)[source]¶ Computes MISH(A Self Regularized Non-Monotonic Neural Activation Function) of input tensors element-wise.
The function is shown as follows:
\[\text{output} = x * \tan(\log(1 + \exp(\text{x})))\]See more details in A Self Regularized Non-Monotonic Neural Activation Function.
- Inputs:
x (Tensor) - The input tensor. Only support float16 and float32.
- Outputs:
Tensor, with the same type and shape as the x.
- Supported Platforms:
Ascend- Raise:
TypeError: If dtype of x is neither float16 nor float32.
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> mish = ops.Mish() >>> output = mish(input_x) >>> print(output) [[-3.034014e-01 3.997413e+00 -2.682209e-03] [ 1.943959e+00 -3.357619e-02 8.999999e+00]]
-
class
tinyms.primitives.Mod(*args, **kwargs)[source]¶ Computes the remainder of dividing the first input tensor by the second input tensor element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, both dtypes cannot be bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number]) - The first input is a number or a tensor whose data type is number.
input_y (Union[Tensor, Number]) - When the first input is a tensor, The second input could be a number or a tensor whose data type is number. When the first input is a number, the second input must be a tensor whose data type is number.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
ValueError – When input_x and input_y are not the same dtype.
- Supported Platforms:
AscendCPU
Examples
>>> input_x = Tensor(np.array([-4.0, 5.0, 6.0]), mindspore.float32) >>> input_y = Tensor(np.array([3.0, 2.0, 3.0]), mindspore.float32) >>> mod = ops.Mod() >>> output = mod(input_x, input_y) >>> print(output) [-1. 1. 0.]
-
class
tinyms.primitives.Mul(*args, **kwargs)[source]¶ Multiplies two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
\[out_{i} = x_{i} * y_{i}\]- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
ValueError – If input_x and input_y are not the same shape.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([4.0, 5.0, 6.0]), mindspore.float32) >>> mul = ops.Mul() >>> output = mul(input_x, input_y) >>> print(output) [ 4. 10. 18.]
-
class
tinyms.primitives.MulNoNan(*args, **kwargs)[source]¶ Computes input_x * input_y element-wise. If input_y is zero, no matter what input_x is, it will return 0.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, the shapes of them could be broadcasted. When the inputs are one tensor and one scalar, the scalar could only be a constant.
Note
The shapes of input_x and input_y should be the same or can be broadcasted.
- Inputs:
input_x (Union[Tensor]) - The first input is a tensor whose data type is one of flota16, float32, int32, int64 currently or scalar.
input_y (Union[Tensor]) - The second input is a tensor whose data type is one of flota16, float32, int32, int64 currently or scalar.
- Outputs:
Tensor, the shape is the same as the shape after broadcasting, and the data type is the one with higher precision among the two inputs.
- Supported Platforms:
Ascend
- Raises
TypeError – If neither input_x nor input_y is a bool Tensor.
Examples
>>> x = Tensor(np.array([[-1.0, 6.0, np.inf], [np.nan, -7.0, 4.0]]), ms.float32) >>> y = Tensor(np.array([[-1.0, 4.0, 0], [0, -3.0, 1.0]]), ms.float32) >>> mul_no_nan = ops.MulNoNan() >>> output = mul_no_nan(x, y) >>> print(output) [[ 1. 24. 0.] [ 0. 21. 4.]]
-
class
tinyms.primitives.Multinomial(*args, **kwargs)[source]¶ Returns a tensor sampled from the multinomial probability distribution located in the corresponding row of tensor input.
Note
The rows of input do not need to sum to one (in which case we use the values as weights), but must be non-negative, finite and have a non-zero sum.
- Parameters
- Inputs:
input (Tensor[float32]) - the input tensor containing the cumsum of probabilities, must be 1 or 2 dimensions.
num_samples (int32) - number of samples to draw.
- Outputs:
Tensor with the same rows as input, each row has num_samples sampled indices.
- Raises
- Supported Platforms:
GPU
Examples
>>> input = Tensor([0., 9., 4., 0.], mstype.float32) >>> multinomial = ops.Multinomial(seed=10) >>> output = multinomial(input, 2) >>> print(output) [2 1]
-
class
tinyms.primitives.NLLLoss(*args, **kwargs)[source]¶ Gets the negative log likelihood loss between logits and labels.
The nll loss with reduction=none can be described as:
\[\ell(x, t)=L=\left\{l_{1}, \ldots, l_{N}\right\}^{\top}, \quad l_{n}=-w_{t_{n}} x_{n, t_{n}}, \quad w_{c}=\text { weight }[c] \cdot 1\]where x is the input, t is the target. w is the weight. and N is the batch size. c belonging [0, C-1] is class index, where C is the number of classes.
If reduction is not ‘none’ (default ‘mean’), then
\[\begin{split}\ell(x, t)=\left\{\begin{array}{ll} \sum_{n=1}^{N} \frac{1}{\sum_{n=1}^{N} w_{t n}} l_{n}, & \text { if reduction }=\text { 'mean'; } \\ \sum_{n=1}^{N} l_{n}, & \text { if reduction }=\text { 'sum' } \end{array}\right.\end{split}\]- Parameters
reduction (string) – Apply specific reduction method to the output: ‘none’, ‘mean’, ‘sum’. Default: “mean”.
- Inputs:
input (Tensor) - Input logits, with shape \((N, C)\). Data type only support float32 or float16.
target (Tensor) - Ground truth labels, with shape \((N)\). Data type only support int32.
weight (Tensor) - The rescaling weight to each class, with shape \((C)\) and data type only support float32 or float16`.
- Outputs:
Tuple of 2 tensors composed with loss and total_weight.
loss (Tensor) - when reduction is none and input is 2D tensor, the loss shape is (N,). Otherwise, the loss is a scalar. The data type is same with input’s.
total_weight (Tensor) - the total_weight is a scalar. The data type is same with weight’s.
- Raises
TypeError – If x and weight data type are not float16 or float32 tensor, target data type is not int32 tensor.
ValueError – If x is not a one or two dimension tensor, target and weight not a one dimension tensor. When x is a two dimension tensor, the first dimension of x is not equal to target, and second dimension of x is not equal to weight. When x is a one dimension tensor, the dimensions of x, target and weight should be equal to each other.
- Supported Platforms:
Ascend
Examples
>>> input = Tensor(np.array([[0.5488135, 0.71518934], ... [0.60276335, 0.5448832], ... [0.4236548, 0.6458941]]).astype(np.float32)) >>> target = Tensor(np.array([0, 0, 0]).astype(np.int32)) >>> weight = Tensor(np.array([0.3834415, 0.79172504]).astype(np.float32)) >>> nll_loss = ops.NLLLoss(reduction="mean") >>> loss, weight = nll_loss(input, target, weight) >>> print(loss) -0.52507716 >>> print(weight) 1.1503246
-
class
tinyms.primitives.NMSWithMask(*args, **kwargs)[source]¶ When object detection problem is performed in the computer vision field, object detection algorithm generates a plurality of bounding boxes. Selects some bounding boxes in descending order of score. Use the box with the highest score calculate the overlap between other boxes and the current box, and delete the box based on a certain threshold(IOU). The IOU is as follows,
\[\text{IOU} = \frac{\text{Area of Overlap}}{\text{Area of Union}}\]- Parameters
iou_threshold (float) – Specifies the threshold of overlap boxes with respect to IOU. Default: 0.5.
- Inputs:
bboxes (Tensor) - The shape of tensor is \((N, 5)\). Input bounding boxes. N is the number of input bounding boxes. Every bounding box contains 5 values, the first 4 values are the coordinates(x0, y0, x1, y1) of bounding box which represents the point of top-left and bottom-right, and the last value is the score of this bounding box. The data type must be float16 or float32.
- Outputs:
tuple[Tensor], tuple of three tensors, they are selected_boxes, selected_idx and selected_mask.
selected_boxes (Tensor) - The shape of tensor is \((N, 5)\). The list of bounding boxes after non-max suppression calculation.
selected_idx (Tensor) - The shape of tensor is \((N,)\). The indexes list of valid input bounding boxes.
selected_mask (Tensor) - The shape of tensor is \((N,)\). A mask list of valid output bounding boxes.
- Raises
ValueError – If the iou_threshold is not a float number, or if the first dimension of input Tensor is less than or equal to 0, or if the data type of the input Tensor is not float16 or float32.
- Supported Platforms:
AscendGPU
Examples
>>> bbox = np.array([[100.0, 100.0, 50.0, 68.0, 0.63], [150.0, 75.0, 165.0, 115.0, 0.55], ... [12.0, 190.0, 288.0, 200.0, 0.9], [28.0, 130.0, 106.0, 172.0, 0.3]]) >>> bbox[:, 2] += bbox[:, 0] >>> bbox[:, 3] += bbox[:, 1] >>> inputs = Tensor(bbox, mindspore.float32) >>> nms = ops.NMSWithMask(0.1) >>> output_boxes, indices, mask = nms(inputs) >>> indices_np = indices.asnumpy() >>> print(indices_np[mask.asnumpy()]) [0 1 2]
-
class
tinyms.primitives.NPUAllocFloatStatus(*args, **kwargs)[source]¶ Allocates a flag to store the overflow status.
The flag is a tensor whose shape is (8,) and data type is mindspore.dtype.float32.
Note
Examples: see NPUGetFloatStatus.
- Outputs:
Tensor, has the shape of (8,).
- Supported Platforms:
Ascend
Examples
>>> alloc_status = ops.NPUAllocFloatStatus() >>> output = alloc_status() >>> print(output) [0. 0. 0. 0. 0. 0. 0. 0.]
-
class
tinyms.primitives.NPUClearFloatStatus(*args, **kwargs)[source]¶ Clears the flag which stores the overflow status.
Note
The flag is in the register on the Ascend device. It will be reset and can not be reused again after the NPUClearFloatStatus is called.
Examples: see NPUGetFloatStatus.
- Inputs:
input_x (Tensor) - The output tensor of NPUAllocFloatStatus. The data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x. All the elements in the tensor will be zero.
- Supported Platforms:
Ascend
Examples
>>> alloc_status = ops.NPUAllocFloatStatus() >>> get_status = ops.NPUGetFloatStatus() >>> clear_status = ops.NPUClearFloatStatus() >>> init = alloc_status() >>> flag = get_status(init) >>> clear_status(init) Tensor(shape=[8], dtype=Float32, value= [ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00]) >>> print(init) [1. 1. 1. 1. 1. 1. 1. 1.]
-
class
tinyms.primitives.NPUGetFloatStatus(*args, **kwargs)[source]¶ Updates the flag which is the output tensor of NPUAllocFloatStatus with the latest overflow status.
The flag is a tensor whose shape is (8,) and data type is mindspore.dtype.float32. If the sum of the flag equals to 0, there is no overflow happened. If the sum of the flag is bigger than 0, there is overflow happened.
- Inputs:
input_x (Tensor) - The output tensor of NPUAllocFloatStatus. The data type must be float16 or float32.
- Outputs:
Tensor, has the same shape as input_x. All the elements in the tensor will be zero.
- Raises
- Supported Platforms:
Ascend
Examples
>>> alloc_status = ops.NPUAllocFloatStatus() >>> get_status = ops.NPUGetFloatStatus() >>> init = alloc_status() >>> get_status(init) Tensor(shape=[8], dtype=Float32, value= [ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00]) >>> print(init) [1. 1. 1. 1. 1. 1. 1. 1.]
-
class
tinyms.primitives.Neg(*args, **kwargs)[source]¶ Returns a tensor with negative values of the input tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor whose dtype is number.
- Outputs:
Tensor, has the same shape and dtype as input.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> neg = ops.Neg() >>> input_x = Tensor(np.array([1, 2, -1, 2, 0, -3.5]), mindspore.float32) >>> output = neg(input_x) >>> print(output) [-1. -2. 1. -2. 0. 3.5]
-
class
tinyms.primitives.NeighborListUpdate(*args, **kwargs)[source]¶ Update (or construct if first time) the Verlet neighbor list for the calculation of short-ranged force. Assume the number of atoms is N, the number of grids divided is G, the maximum number of atoms in one grid is M, the maximum number of atoms in single atom’s neighbor list is L, and the number of total atom in excluded list is E.
- Parameters
grid_numbers (int32) – the total number of grids divided.
not_first_time (int32) – whether to construct the neighbor list first time or not.
Nxy (int32) – the total number of grids divided in xy plane.
excluded_atom_numbers (int32) – the total atom numbers in the excluded list.
cutoff (float32) – the cutoff distance for short-range force calculation.
skin (float32) – the overflow value of cutoff to maintain a neighbor list.
cutoff_square (float32) – the suqare value of cutoff.
half_skin_square (float32) – skin*skin/4, indicates the maximum square value of the distance atom allowed to move between two updates.
cutoff_with_skin (float32) – cutoff + skin, indicates the radius of the neighbor list for each atom.
half_cutoff_with_skin (float32) – cutoff_with_skin/2.
cutoff_with_skin_square (float32) – the square value of cutoff_with_skin.
refresh_interval (int32) – the number of iteration steps between two updates of neighbor list.
max_atom_in_grid_numbers (int32) – the maximum number of atoms in one grid.
- Inputs:
atom_numbers_in_grid_bucket (Tensor, int32) - [G,], the number of atoms in each grid bucket.
bucket (Tensor, int32) - (Tensor,int32) - [G, M], the atom indices in each grid bucket.
crd (Tensor, float32) - [N,], the coordinates of each atom.
box_length (Tensor, float32) - [3,], the length of 3 dimensions of the simulation box.
grid_N (Tensor, int32) - [3,], the number of grids divided of 3 dimensions of the simulation box.
grid_length_inverse (float32) - the inverse value of grid length.
atom_in_grid_serial (Tensor, int32) - [N,], the grid index for each atom.
old_crd (Tensor, float32) - [N, 3], the coordinates before update of each atom.
crd_to_uint_crd_cof (Tensor, float32) - [3,], the scale factor between the unsigned int value and the real space coordinates.
uint_crd (Tensor, uint32) - [N, 3], the unsigned int coordinates value fo each atom.
gpointer (Tensor, int32) - [G, 125], the 125 nearest neighbor grids (including self) of each grid. G is the number of nearest neighbor grids.
nl_atom_numbers (Tensor, int32) - [N,], the number of atoms in neighbor list of each atom.
nl_atom_serial (Tensor, int32) - [N, L], the indices of atoms in neighbor list of each atom.
uint_dr_to_dr_cof (Tensor, float32) - [3,], the scale factor between the real space coordinates and the unsigned int value.
excluded_list_start (Tensor, int32) - [N,], the start excluded index in excluded list for each atom.
excluded_numbers (Tensor, int32) - [N,], the number of atom excluded in excluded list for each atom.
excluded_list (Tensor, int32) - [E,], the contiguous join of excluded list of each atom.
need_refresh_flag (Tensor, int32) - [N,], whether the neighbor list of each atom need update or not.
refresh_count (Tensor, int32) - [1,], count how many iteration steps have passed since last update.
- Outputs:
res (float32)
- Supported Platforms:
GPU
-
class
tinyms.primitives.NoRepeatNGram(*args, **kwargs)[source]¶ Updates log_probs with repeat n-grams.
During beam search, if consecutive ngram_size words exist in the generated word sequence, the consecutive ngram_size words will be avoided during subsequent prediction. For example, when ngram_size is 3, the generated word sequence is [1, 2, 3, 2, 3], the next predicted word will not be 2 and the value of log_probs will be replaced with -FLOAT_MAX. Because 3 consecutive words [2, 3, 2] do not appear twice in the word sequence.
- Parameters
ngram_size (int) – Size of n-grams, must be greater than 0. Default: 1.
- Inputs:
state_seq (Tensor) - A 3-D tensor with shape: (batch_size, beam_width, m).
log_probs (Tensor) - A 3-D tensor with shape: (batch_size, beam_width, vocab_size). The value of log_probs will be replaced with -FLOAT_MAX when n-grams repeated.
- Outputs:
log_probs (Tensor) - The output Tensor with same shape and type as original log_probs.
- Raises
- Supported Platforms:
Ascend
Examples
>>> no_repeat_ngram = ops.NoRepeatNGram(ngram_size=3) >>> state_seq = Tensor([[[1, 2, 1, 2, 5, 1, 2], ... [9, 3, 9, 5, 4, 1, 5]], ... [[4, 8, 6, 4, 5, 6, 4], ... [4, 8, 8, 4, 3, 4, 8]]], dtype=mindspore.int32) >>> log_probs = Tensor([[[0.7, 0.8, 0.6, 0.9, 0.2, 0.8, 0.4, 0.6, 0.2, 0.7], ... [0.4, 0.5, 0.6, 0.7, 0.8, 0.1, 0.9, 0.8, 0.7, 0.1]], ... [[0.9, 0.7, 0.6, 0.3, 0.5, 0.3, 0.5, 0.4, 0.8, 0.6], ... [0.5, 0.8, 0.8, 0.7, 0.7, 0.8, 0.2, 0.7, 0.9, 0.7]]], dtype=mindspore.float32) >>> output = no_repeat_ngram(state_seq, log_probs) >>> print(output) [[[ 6.9999999e-01 -3.4028235e+38 6.0000002e-01 8.9999998e-01 2.0000000e-01 -3.4028235e+38 4.0000001e-01 6.0000002e-01 2.0000000e-01 6.9999999e-01] [ 4.0000001e-01 5.0000000e-01 6.0000002e-01 6.9999999e-01 8.0000001e-01 1.0000000e-01 8.9999998e-01 8.0000001e-01 6.9999999e-01 1.0000000e-01]] [[ 8.9999998e-01 6.9999999e-01 6.0000002e-01 3.0000001e-01 5.0000000e-01 -3.4028235e+38 5.0000000e-01 4.0000001e-01 8.0000001e-01 6.0000002e-01] [ 5.0000000e-01 8.0000001e-01 8.0000001e-01 6.9999999e-01 6.9999999e-01 8.0000001e-01 2.0000000e-01 6.9999999e-01 -3.4028235e+38 6.9999999e-01]]]
-
class
tinyms.primitives.NotEqual(*args, **kwargs)[source]¶ Computes the non-equivalence of two tensors element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting,and the data type is bool.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> not_equal = ops.NotEqual() >>> output = not_equal(input_x, 2.0) >>> print(output) [ True False True] >>> >>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([1, 2, 4]), mindspore.int32) >>> not_equal = ops.NotEqual() >>> output = not_equal(input_x, input_y) >>> print(output) [False False True]
-
class
tinyms.primitives.OneHot(*args, **kwargs)[source]¶ Computes a one-hot tensor.
Makes a new tensor, whose locations represented by indices in indices take value on_value, while all other locations take value off_value.
Note
If the input indices is rank N, the output will have rank N+1. The new axis is created at dimension axis.
- Parameters
axis (int) – Position to insert the value. e.g. If indices shape is [n, c], and axis is -1 the output shape will be [n, c, depth], If axis is 0 the output shape will be [depth, n, c]. Default: -1.
- Inputs:
indices (Tensor) - A tensor of indices. Tensor of shape \((X_0, \ldots, X_n)\). Data type must be int32 or int64.
depth (int) - A scalar defining the depth of the one hot dimension.
on_value (Tensor) - A value to fill in output when indices[j] = i. With data type of float16 or float32.
off_value (Tensor) - A value to fill in output when indices[j] != i. Has the same data type with as on_value.
- Outputs:
Tensor, one-hot tensor. Tensor of shape \((X_0, \ldots, X_{axis}, \text{depth} ,X_{axis+1}, \ldots, X_n)\).
- Raises
TypeError – If axis or depth is not an int.
TypeError – If dtype of indices is neither int32 nor int64.
TypeError – If indices, on_value or off_value is not a Tensor.
ValueError – If axis is not in range [-1, len(indices_shape)].
ValueError – If depth is less than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> indices = Tensor(np.array([0, 1, 2]), mindspore.int32) >>> depth, on_value, off_value = 3, Tensor(1.0, mindspore.float32), Tensor(0.0, mindspore.float32) >>> onehot = ops.OneHot() >>> output = onehot(indices, depth, on_value, off_value) >>> print(output) [[1. 0. 0.] [0. 1. 0.] [0. 0. 1.]]
-
class
tinyms.primitives.Ones(*args, **kwargs)[source]¶ Creates a tensor filled with value ones.
Creates a tensor with shape described by the first argument and fills it with value ones in type of the second argument.
- Inputs:
shape (Union[tuple[int], int]) - The specified shape of output tensor. Only constant positive int is allowed.
type (mindspore.dtype) - The specified type of output tensor. Only constant value is allowed.
- Outputs:
Tensor, has the same type and shape as input shape value.
- Raises
TypeError – If shape is neither tuple nor int.
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore.ops import operations as ops >>> ones = ops.Ones() >>> output = ones((2, 2), mindspore.float32) >>> print(output) [[1. 1.] [1. 1.]]
-
class
tinyms.primitives.OnesLike(*args, **kwargs)[source]¶ Creates a new tensor. The values of all elements are 1.
Returns a tensor of ones with the same shape and type as the input.
- Inputs:
input_x (Tensor) - Input tensor.
- Outputs:
Tensor, has the same shape and type as input_x but filled with ones.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> oneslike = ops.OnesLike() >>> x = Tensor(np.array([[0, 1], [2, 1]]).astype(np.int32)) >>> output = oneslike(x) >>> print(output) [[1 1] [1 1]]
-
class
tinyms.primitives.PMEEnergy(*args, **kwargs)[source]¶ Calculate the Coulumb energy of the system using PME method.
\[E = sum_{ij} q_iq_j/r_{ij}\]- Parameters
atom_numbers (int32) – the number of atoms, N.
excluded_numbers (int32) – the length of excluded list, E.
beta (float32) – the PME beta parameter, determined by the non-bond cutoff value and simulation precision tolerance.
fftx (int32) – the number of points for Fourier transform in dimension X.
ffty (int32) – the number of points for Fourier transform in dimension Y.
fftz (int32) – the number of points for Fourier transform in dimension Z.
box_length_0 (float32) – the value of boxlength idx 0
box_length_1 (float32) – the value of boxlength idx 1
box_length_2 (float32) – the value of boxlength idx 2
- Inputs:
uint_crd (Tensor, uint32) - [N, 3], the unsigned int coordinates value of each atom.
charge (Tensor, float32) - [N,], the charge carried by each atom.
nl_numbers - (Tensor, int32) - [N,], the each atom.
nl_serial - (Tensor, int32) - [N, 800], the neighbor list of each atom, the max number is 800.
scaler (Tensor, float32) - [3,], the scale factor between real space coordinates and its unsigned int value.
excluded_list_start (Tensor, int32) - [N,], the start excluded index in excluded list for each atom.
excluded_list (Tensor, int32) - [E,], the contiguous join of excluded list of each atom. E is the number of excluded atoms.
excluded_atom_numbers (Tensor, int32) - [N,], the number of atom excluded in excluded list for each atom.
- Outputs:
reciprocal_ene (float32) - the reciprocal term of PME energy.
self_ene (float32) - the self term of PME energy.
direct_ene (float32) - the direct term of PME energy.
correction_ene (float32) - the correction term of PME energy.
- Supported Platforms:
GPU
-
class
tinyms.primitives.PMEExcludedForce(*args, **kwargs)[source]¶ Calculate the excluded part of long-range Coulumb force using PME(Particle Meshed Ewald) method. Assume the number of atoms is N, and the length of excluded list is E.
- Parameters
atom_numbers (int32) – the number of atoms, N.
excluded_numbers (int32) – the length of excluded list, E.
beta (float32) – the PME beta parameter, determined by the non-bond cutoff value and simulation precision tolerance.
- Inputs:
uint_crd (Tensor, uint32) - [N, 3], the unsigned int coordinates value of each atom.
scaler (Tensor, float32) - [3,], the scale factor between real space coordinates and its unsigned int value.
charge (Tensor, float32) - [N,], the charge carried by each atom.
excluded_list_start (Tensor, int32) - [N,], the start excluded index in excluded list for each atom.
excluded_list (Tensor, int32) - [E,], the contiguous join of excluded list of each atom. E is the number of excluded atoms.
excluded_atom_numbers (Tensor, int32) - [N,], the number of atom excluded in excluded list for each atom.
- Outputs:
force (Tensor, float32) - [N, 3], the force felt by each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.PMEReciprocalForce(*args, **kwargs)[source]¶ Calculate the reciprocal part of long-range Coulumb force using PME(Particle Meshed Ewald) method. Assume the number of atoms is N.
The detailed calculation formula of PME(Particle Meshed Ewald) method can be found in this paper: A Smooth Particle Mesh Ewald Method. DOI: 10.1063/1.470117.
- Parameters
atom_numbers (int32) – the number of atoms, N.
beta (float32) – the PME beta parameter, determined by the non-bond cutoff value and simulation precision tolerance.
fftx (int32) – the number of points for Fourier transform in dimension X.
ffty (int32) – the number of points for Fourier transform in dimension Y.
fftz (int32) – the number of points for Fourier transform in dimension Z.
box_length_0 (float32) – the value of boxlength idx 0
box_length_1 (float32) – the value of boxlength idx 1
box_length_2 (float32) – the value of boxlength idx 2
- Inputs:
uint_crd (Tensor, uint32) - [N, 3], the unsigned int coordinates value of each atom.
charge (Tensor, float32) - [N,], the charge carried by each atom.
- Outputs:
force (Tensor, float32) - [N, 3], the force felt by each atom.
- Supported Platforms:
GPU
-
class
tinyms.primitives.PQC(*args, **kwargs)[source]¶ Evaluate a parameterized quantum circuit and calculate the gradient of each parameters.
Inputs of this operation is generated by MindQuantum framework.
- Inputs:
n_qubits (int) - The qubit number of quantum simulator.
encoder_params_names (List[str]) - The parameters names of encoder circuit.
ansatz_params_names (List[str]) - The parameters names of ansatz circuit.
gate_names (List[str]) - The name of each gate.
- gate_matrix (List[List[List[List[float]]]]) - Real part and image
part of the matrix of quantum gate.
gate_obj_qubits (List[List[int]]) - Object qubits of each gate.
gate_ctrl_qubits (List[List[int]]) - Control qubits of each gate.
gate_params_names (List[List[str]]) - Parameter names of each gate.
gate_coeff (List[List[float]]) - Coefficient of eqch parameter of each gate.
- gate_requires_grad (List[List[bool]]) - Whether to calculate gradient
of parameters of gates.
hams_pauli_coeff (List[List[float]]) - Coefficient of pauli words.
hams_pauli_word (List[List[List[str]]]) - Pauli words.
hams_pauli_qubit (List[List[List[int]]]) - The qubit that pauli matrix act on.
n_threads (int) - Thread to evaluate input data.
- Outputs:
expected_value (Tensor) - The expected value of hamiltonian.
g1 (Tensor) - Gradient of encode circuit parameters.
g2 (Tensor) - Gradient of ansatz circuit parameters.
- Supported Platforms:
CPU
-
class
tinyms.primitives.PReLU(*args, **kwargs)[source]¶ Parametric Rectified Linear Unit activation function.
PReLU is described in the paper Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Defined as follows:
\[prelu(x_i)= \max(0, x_i) + \min(0, w * x_i),\]where \(x_i\) is an element of an channel of the input.
Note
1-dimensional input_x is not supported.
- Inputs:
input_x (Tensor) - Float tensor, representing the output of the preview layer. With data type of float16 or float32.
weight (Tensor) - Float Tensor, w > 0, there are only two shapes are legitimate, 1 or the number of channels of the input. With data type of float16 or float32.
- Outputs:
Tensor, with the same type as input_x.
For detailed information, please refer to nn.PReLU.
- Raises
TypeError – If dtype of input_x or weight is neither float16 nor float32.
TypeError – If input_x or weight is not a Tensor.
ValueError – If length of shape of input_x is equal to 1.
ValueError – If length of shape of weight is not equal to 1.
- Supported Platforms:
Ascend
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.prelu = ops.PReLU() ... def construct(self, input_x, weight): ... result = self.prelu(input_x, weight) ... return result ... >>> input_x = Tensor(np.random.randint(-3, 3, (2, 3, 2)), mindspore.float32) >>> weight = Tensor(np.array([0.1, 0.6, -0.3]), mindspore.float32) >>> net = Net() >>> output = net(input_x, weight) >>> print(output.shape) (2, 3, 2)
-
class
tinyms.primitives.Pack(**kwargs)[source]¶ Same as operator Stack. Pack will be deprecated in the future. Please use Stack instead.
-
class
tinyms.primitives.Pad(*args, **kwargs)[source]¶ Pads the input tensor according to the paddings.
- Parameters
paddings (tuple) – The shape of parameter paddings is (N, 2). N is the rank of input data. All elements of paddings are int type. For the input in D th dimension, paddings[D, 0] indicates how many sizes to be extended ahead of the input tensor in the D th dimension, and paddings[D, 1] indicates how many sizes to be extended behind the input tensor in the D th dimension.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, the tensor after padding.
- Raises
TypeError – If paddings is not a tuple.
TypeError – If input_x is not a Tensor.
ValueError – If shape of paddings is not (n, 2).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]), mindspore.float32) >>> pad_op = ops.Pad(((1, 2), (2, 1))) >>> output = pad_op(input_tensor) >>> print(output) [[ 0. 0. 0. 0. 0. 0. ] [ 0. 0. -0.1 0.3 3.6 0. ] [ 0. 0. 0.4 0.5 -3.2 0. ] [ 0. 0. 0. 0. 0. 0. ] [ 0. 0. 0. 0. 0. 0. ]]
-
class
tinyms.primitives.Padding(*args, **kwargs)[source]¶ Extends the last dimension of the input tensor from 1 to pad_dim_size, by filling with 0.
- Parameters
pad_dim_size (int) – The value of the last dimension of x to be extended, which must be positive.
- Inputs:
x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The rank of x must be at least 2. The last dimension of x must be 1.
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Raises
TypeError – If pad_dim_size is not an int.
ValueError – If pad_dim_size is less than 1.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([[8], [10]]), mindspore.float32) >>> pad_dim_size = 4 >>> output = ops.Padding(pad_dim_size)(x) >>> print(output) [[ 8. 0. 0. 0.] [10. 0. 0. 0.]]
-
class
tinyms.primitives.ParallelConcat(*args, **kwargs)[source]¶ Concats tensor in the first dimension.
Concats input tensors along with the first dimension.
Note
The input tensors are all required to have size 1 in the first dimension.
- Inputs:
values (tuple, list) - A tuple or a list of input tensors. The data type and shape of these tensors must be the same.
- Outputs:
Tensor, data type is the same as values.
- Raises
ValueError – If length of shape of values is less than 1.
- Supported Platforms:
Ascend
Examples
>>> data1 = Tensor(np.array([[0, 1]]).astype(np.int32)) >>> data2 = Tensor(np.array([[2, 1]]).astype(np.int32)) >>> op = ops.ParallelConcat() >>> output = op((data1, data2)) >>> print(output) [[0 1] [2 1]]
-
class
tinyms.primitives.Partial(*args, **kwargs)[source]¶ Makes a partial function instance, used for pynative mode.
- Inputs:
args (Union[FunctionType, Tensor]) - The function and bind arguments.
- Outputs:
FunctionType, partial function binded with arguments.
-
class
tinyms.primitives.Poisson(*args, **kwargs)[source]¶ Produces random non-negative integer values i, distributed according to discrete probability function:
\[\text{P}(i|μ) = \frac{\exp(-μ)μ^{i}}{i!},\]- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
mean (Tensor) - μ parameter the distribution was constructed with. The parameter defines mean number of occurrences of the event. It must be greater than 0. With float32 data type.
- Outputs:
Tensor. Its shape must be the broadcasted shape of shape and the shape of mean. The dtype is int32.
- Raises
- Supported Platforms:
Ascend
Examples
>>> shape = (4, 1) >>> mean = Tensor(np.array([5.0, 10.0]), mstype.float32) >>> poisson = ops.Poisson(seed=5) >>> output = poisson(shape, mean) >>> result = output.shape >>> print(result) (4, 2)
-
class
tinyms.primitives.PopulationCount(*args, **kwargs)[source]¶ Calculates population count.
- Inputs:
input (Tensor) - The data type must be int16 or uint16.
- Outputs:
Tensor, with the same shape as the input.
- Raises
TypeError – If input is not a Tensor.
- Supported Platforms:
Ascend
Examples
>>> population_count = ops.PopulationCount() >>> x_input = Tensor([0, 1, 3], mindspore.int16) >>> output = population_count(x_input) >>> print(output) [0 1 2]
-
class
tinyms.primitives.Pow(*args, **kwargs)[source]¶ Computes a tensor to the power of the second input.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
ValueError – If input_x and input_y are not the same shape.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> input_y = 3.0 >>> pow = ops.Pow() >>> output = pow(input_x, input_y) >>> print(output) [ 1. 8. 64.] >>> >>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> input_y = Tensor(np.array([2.0, 4.0, 3.0]), mindspore.float32) >>> pow = ops.Pow() >>> output = pow(input_x, input_y) >>> print(output) [ 1. 16. 64.]
-
class
tinyms.primitives.Print(*args, **kwargs)[source]¶ Outputs the tensor or string to stdout.
Note
In pynative mode, please use python print function. In graph mode, the bool, int and float would be converted into Tensor to print, str remains unchanged.
- Inputs:
input_x (Union[Tensor, bool, int, float, str]) - The graph node to attach to. Supports multiple inputs which are separated by ‘,’.
- Raises
TypeError – If input_x is not one of the following: Tensor, bool, int, float, str.
- Supported Platforms:
AscendGPU
Examples
>>> class PrintDemo(nn.Cell): ... def __init__(self): ... super(PrintDemo, self).__init__() ... self.print = ops.Print() ... ... def construct(self, x, y): ... self.print('Print Tensor x and Tensor y:', x, y) ... return x ... >>> x = Tensor(np.ones([2, 1]).astype(np.int32)) >>> y = Tensor(np.ones([2, 2]).astype(np.int32)) >>> net = PrintDemo() >>> result = net(x, y) Print Tensor x and Tensor y: [[1] [1]] [[1 1] [1 1]]
-
class
tinyms.primitives.Pull(*args, **kwargs)[source]¶ Pulls weight from parameter server.
- Inputs:
key (Tensor) - The key of the weight.
weight (Tensor) - The weight to be updated.
- Outputs:
None.
-
class
tinyms.primitives.Push(*args, **kwargs)[source]¶ Pushes the inputs of the corresponding optimizer to parameter server.
- Parameters
optim_type (string) – The optimizer type. Default: ‘ApplyMomentum’.
only_shape_indices (list) – The indices of input of which only shape will be pushed to parameter server. Default: None.
- Inputs:
optim_inputs (tuple) - The inputs for this kind of optimizer.
optim_input_shapes (tuple) - The shapes of the inputs.
- Outputs:
Tensor, the key of the weight which needs to be updated.
-
class
tinyms.primitives.RNNTLoss(*args, **kwargs)[source]¶ Computes the RNNTLoss and its gradient with respect to the softmax outputs.
- Parameters
blank_label (int) – blank label. Default: 0.
- Inputs:
acts (Tensor) - Tensor of shape \((B, T, U, V)\). Data type must be float16 or float32.
labels (Tensor[int32]) - Tensor of shape \((B, U-1)\).
input_lengths (Tensor[int32]) - Tensor of shape \((B,)\).
label_lengths (Tensor[int32]) - Tensor of shape \((B,)\).
- Outputs:
costs (Tensor[int32]) - Tensor of shape \((B,)\).
grads (Tensor[int32]) - Has the same shape as acts.
- Raises
- Supported Platforms:
Ascend
Examples
>>> B, T, U, V = 1, 2, 3, 5 >>> blank = 0 >>> acts = np.random.random((B, T, U, V)).astype(np.float32) >>> labels = np.array([[1, 2]]).astype(np.int32) >>> input_length = np.array([T] * B).astype(np.int32) >>> label_length = np.array([len(l) for l in labels]).astype(np.int32) >>> rnnt_loss = ops.RNNTLoss(blank_label=0) >>> costs, grads = rnnt_loss(Tensor(acts), Tensor(labels), Tensor(input_length), Tensor(label_length)) >>> print(costs.shape) (1,) >>> print(grads.shape) (1, 2, 3, 5)
-
class
tinyms.primitives.ROIAlign(*args, **kwargs)[source]¶ Computes the Region of Interest (RoI) Align operator.
The operator computes the value of each sampling point by bilinear interpolation from the nearby grid points on the feature map. No quantization is performed on any coordinates involved in the RoI, its bins, or the sampling points. The details of (RoI) Align operator are described in Mask R-CNN.
- Parameters
pooled_height (int) – The output features height.
pooled_width (int) – The output features width.
spatial_scale (float) – A scaling factor that maps the raw image coordinates to the input feature map coordinates. Suppose the height of a RoI is ori_h in the raw image and fea_h in the input feature map, the spatial_scale must be fea_h / ori_h.
sample_num (int) – Number of sampling points. Default: 2.
roi_end_mode (int) – Number must be 0 or 1. Default: 1.
- Inputs:
features (Tensor) - The input features, whose shape must be (N, C, H, W).
rois (Tensor) - The shape is (rois_n, 5). With data type of float16 or float32. rois_n represents the number of RoI. The size of the second dimension must be 5 and the 5 colunms are (image_index, top_left_x, top_left_y, bottom_right_x, bottom_right_y). image_index represents the index of image. top_left_x and top_left_y represent the x, y coordinates of the top left corner of corresponding RoI, respectively. bottom_right_x and bottom_right_y represent the x, y coordinates of the bottom right corner of corresponding RoI, respectively.
- Outputs:
Tensor, the shape is (rois_n, C, pooled_height, pooled_width).
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> input_tensor = Tensor(np.array([[[[1., 2.], [3., 4.]]]]), mindspore.float32) >>> rois = Tensor(np.array([[0, 0.2, 0.3, 0.2, 0.3]]), mindspore.float32) >>> roi_align = ops.ROIAlign(2, 2, 0.5, 2) >>> output = roi_align(input_tensor, rois) >>> print(output) [[[[1.775 2.025] [2.275 2.525]]]]
-
class
tinyms.primitives.RandomCategorical(*args, **kwargs)[source]¶ Generates random samples from a given categorical distribution tensor.
- Parameters
dtype (mindspore.dtype) – The type of output. Its value must be one of mindspore.int16, mindspore.int32 and mindspore.int64. Default: mindspore.int64.
- Inputs:
logits (Tensor) - The input tensor. 2-D Tensor with shape [batch_size, num_classes].
num_sample (int) - Number of sample to be drawn. Only constant values is allowed.
seed (int) - Random seed. Default: 0. Only constant values is allowed.
- Outputs:
output (Tensor) - The output Tensor with shape [batch_size, num_samples].
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> class Net(nn.Cell): ... def __init__(self, num_sample): ... super(Net, self).__init__() ... self.random_categorical = ops.RandomCategorical(mindspore.int64) ... self.num_sample = num_sample ... def construct(self, logits, seed=0): ... return self.random_categorical(logits, self.num_sample, seed) ... >>> x = np.random.random((10, 5)).astype(np.float32) >>> net = Net(8) >>> output = net(Tensor(x)) >>> result = output.shape >>> print(result) (10, 8)
-
class
tinyms.primitives.RandomChoiceWithMask(*args, **kwargs)[source]¶ Generates a random sample as index tensor with a mask tensor from a given tensor.
The input must be a tensor of rank not less than 1. If its rank is greater than or equal to 2, the first dimension specifies the number of samples. The index tensor and the mask tensor have the fixed shapes. The index tensor denotes the index of the nonzero sample, while the mask tensor denotes which elements in the index tensor are valid.
- Parameters
- Inputs:
input_x (Tensor[bool]) - The input tensor. The input tensor rank must be greater than or equal to 1 and less than or equal to 5.
- Outputs:
Two tensors, the first one is the index tensor and the other one is the mask tensor.
index (Tensor) - The output shape is 2-D.
mask (Tensor) - The output shape is 1-D.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> rnd_choice_mask = ops.RandomChoiceWithMask() >>> input_x = Tensor(np.ones(shape=[240000, 4]).astype(np.bool)) >>> output_y, output_mask = rnd_choice_mask(input_x) >>> result = output_y.shape >>> print(result) (256, 2) >>> result = output_mask.shape >>> print(result) (256,)
-
class
tinyms.primitives.Randperm(*args, **kwargs)[source]¶ Generates n random samples from 0 to n-1 without repeating. If max_length > n, the last max_length-n elements will be filled with pad.
- Parameters
- Inputs:
n (Tensor[int]) - The input tensor with shape: (1,) and the number must be in (0, max_length]. Default: 1.
- Outputs:
output (Tensor) - The output Tensor with shape: (max_length,) and type: dtype.
- Raises
- Supported Platforms:
Ascend
Examples
>>> randperm = ops.Randperm(max_length=30, pad=-1) >>> n = Tensor([20], dtype=mindspore.int32) >>> output = randperm(n) >>> print(output) [15 6 11 19 14 16 9 5 13 18 4 10 8 0 17 2 14 1 12 3 7 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1]
-
class
tinyms.primitives.Range(*args, **kwargs)[source]¶ Creates a sequence of numbers that begins at start and extends by increments of delta up to but not including limit.
The types of all 3 inputs must be the same. The type of the resulting tensor is the same as the type of the inputs.
- Parameters
maxlen (int) – Memory that can fit maxlen many elements will be allocated for the output. Optional, must be positive, defaults to 1000000. If the output has more than maxlen elements, a runtime error will occur.
- Inputs:
start (Tensor) - A scalar Tensor. The first number in the sequence. Must have type: int32 or float32
limit (Tensor) - A scalar Tensor. Upper limit of the sequence, exclusive. Must have type: int32 or float32
delta (Tensor) - A scalar Tensor. Number that increments start. Must have type: int32 or float32
- Outputs:
A 1-D Tensor, with the same type as the inputs.
- Supported Platforms:
GPU
Examples
>>> start = Tensor(0, mstype.int32) >>> limit = Tensor(10, mstype.int32) >>> delta = Tensor(4, mstype.int32) >>> output = ops.Range()(start, limit, delta) >>> print(output) [0, 4, 8]
-
class
tinyms.primitives.Rank(*args, **kwargs)[source]¶ Returns the rank of a tensor.
Returns a 0-D int32 Tensor representing the rank of input; the rank of a tensor is the number of indices required to uniquely select each element of the tensor.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor. 0-D int32 Tensor representing the rank of input, i.e., \(R\).
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> rank = ops.Rank() >>> output = rank(input_tensor) >>> print(output) 2
-
class
tinyms.primitives.ReLU(*args, **kwargs)[source]¶ Computes ReLU (Rectified Linear Unit) of input tensors element-wise.
It returns \(\max(x,\ 0)\) element-wise.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> relu = ops.ReLU() >>> output = relu(input_x) >>> print(output) [[0. 4. 0.] [2. 0. 9.]]
-
class
tinyms.primitives.ReLU6(*args, **kwargs)[source]¶ Computes ReLU (Rectified Linear Unit) upper bounded by 6 of input tensors element-wise.
\[\text{ReLU6}(x) = \min(\max(0,x), 6)\]It returns \(\min(\max(0,x), 6)\) element-wise.
- Inputs:
input_x (Tensor) - The input tensor, with float16 or float32 data type.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> relu6 = ops.ReLU6() >>> result = relu6(input_x) >>> print(result) [[0. 4. 0.] [2. 0. 6.]]
-
class
tinyms.primitives.ReLUV2(*args, **kwargs)[source]¶ Computes ReLU (Rectified Linear Unit) of input tensors element-wise.
It returns \(\max(x,\ 0)\) element-wise.
- Inputs:
input_x (Tensor) - The input tensor must be a 4-D tensor.
- Outputs:
output (Tensor) - Has the same type and shape as the input_x.
mask (Tensor) - A tensor whose data type must be uint8.
- Raises
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[[[1, -2], [-3, 4]], [[-5, 6], [7, -8]]]]), mindspore.float32) >>> relu_v2 = ops.ReLUV2() >>> output, mask= relu_v2(input_x) >>> print(output) [[[[1. 0.] [0. 4.]] [[0. 6.] [7. 0.]]]] >>> print(mask) [[[[[1 0] [2 0]] [[2 0] [1 0]]]]]
-
class
tinyms.primitives.RealDiv(*args, **kwargs)[source]¶ Divides the first input tensor by the second input tensor in floating-point type element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
ValueError – If input_x and input_y are not the same shape.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([4.0, 5.0, 6.0]), mindspore.float32) >>> realdiv = ops.RealDiv() >>> output = realdiv(input_x, input_y) >>> print(output) [0.25 0.4 0.5 ]
-
class
tinyms.primitives.Reciprocal(*args, **kwargs)[source]¶ Returns reciprocal of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape as the input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 4.0]), mindspore.float32) >>> reciprocal = ops.Reciprocal() >>> output = reciprocal(input_x) >>> print(output) [1. 0.5 0.25]
-
class
tinyms.primitives.ReduceAll(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by the “logicalAND” of all elements in the dimension.
The dtype of the tensor to be reduced is bool.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[bool]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, the dtype is bool.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the “logical and” of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Raises
TypeError – If keep_dims is not a bool.
TypeError – If input_x is not a Tensor.
ValueError – If axis is not one of the following: int, tuple or list.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[True, False], [True, True]])) >>> op = ops.ReduceAll(keep_dims=True) >>> output = op(input_x, 1) >>> print(output) [[False] [ True]]
-
class
tinyms.primitives.ReduceAny(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by the “logical OR” of all elements in the dimension.
The dtype of the tensor to be reduced is bool.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[bool]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, the dtype is bool.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the “logical or” of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Raises
TypeError – If keep_dims is not a bool.
TypeError – If input_x is not a Tensor.
ValueError – If axis is not one of the following: int, tuple or list.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[True, False], [True, True]])) >>> op = ops.ReduceAny(keep_dims=True) >>> output = op(input_x, 1) >>> print(output) [[ True] [ True]]
-
class
tinyms.primitives.ReduceMax(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by the maximum value in this dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the maximum of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Raises
TypeError – If keep_dims is not a bool.
TypeError – If input_x is not a Tensor.
ValueError – If axis is not one of the following: int, tuple or list.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceMax(keep_dims=True) >>> output = op(input_x, 1) >>> result = output.shape >>> print(result) (3, 1, 5, 6)
-
class
tinyms.primitives.ReduceMean(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by averaging all elements in the dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default: False.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the mean of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Raises
TypeError – If keep_dims is not a bool.
TypeError – If input_x is not a Tensor.
ValueError – If axis is not one of the following: int, tuple or list.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceMean(keep_dims=True) >>> output = op(input_x, 1) >>> result = output.shape >>> print(result) (3, 1, 5, 6)
-
class
tinyms.primitives.ReduceMin(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by the minimum value in the dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the minimum of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Raises
TypeError – If keep_dims is not a bool.
TypeError – If input_x is not a Tensor.
ValueError – If axis is not one of the following: int, tuple or list.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceMin(keep_dims=True) >>> output = op(input_x, 1) >>> result = output.shape >>> print(result) (3, 1, 5, 6)
-
class
tinyms.primitives.ReduceOp[source]¶ Operation options for reducing tensors.
There are four kinds of operation options, “SUM”, “MAX”, “MIN”, and “PROD”.
SUM: Take the sum.
MAX: Take the maximum.
MIN: Take the minimum.
PROD: Take the product.
- Supported Platforms:
AscendGPU
-
class
tinyms.primitives.ReduceProd(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by multiplying all elements in the dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default : False, don’t keep these reduced dimensions.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the product of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Raises
TypeError – If keep_dims is not a bool.
TypeError – If input_x is not a Tensor.
ValueError – If axis is not one of the following: int, tuple or list.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceProd(keep_dims=True) >>> output = op(input_x, 1) >>> result = output.shape >>> print(result) (3, 1, 5, 6)
-
class
tinyms.primitives.ReduceScatter(*args, **kwargs)[source]¶ Reduces and scatters tensors from the specified communication group.
Note
The back propagation of the op is not supported yet. Stay tuned for more. The tensors must have the same shape and format in all processes of the collection.
- Parameters
- Raises
TypeError – If any of operation and group is not a string.
ValueError – If the first dimension of the input cannot be divided by the rank size.
- Supported Platforms:
AscendGPU
Examples
>>> # This example should be run with two devices. Refer to the tutorial > Distributed Training on mindspore.cn. >>> from mindspore import Tensor, context >>> from mindspore.communication import init >>> from mindspore.ops.operations.comm_ops import ReduceOp >>> import mindspore.nn as nn >>> import mindspore.ops.operations as ops >>> import numpy as np >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.reducescatter = ops.ReduceScatter(ReduceOp.SUM) ... ... def construct(self, x): ... return self.reducescatter(x) ... >>> input_ = Tensor(np.ones([8, 8]).astype(np.float32)) >>> net = Net() >>> output = net(input_) >>> print(output) [[2. 2. 2. 2. 2. 2. 2. 2.] [2. 2. 2. 2. 2. 2. 2. 2.] [2. 2. 2. 2. 2. 2. 2. 2.] [2. 2. 2. 2. 2. 2. 2. 2.]]
-
class
tinyms.primitives.ReduceSum(*args, **kwargs)[source]¶ Reduces a dimension of a tensor by summing all elements in the dimension.
The dtype of the tensor to be reduced is number.
- Parameters
keep_dims (bool) – If true, keep these reduced dimensions and the length is 1. If false, don’t keep these dimensions. Default: False.
- Inputs:
input_x (Tensor[Number]) - The input tensor.
axis (Union[int, tuple(int), list(int)]) - The dimensions to reduce. Default: (), reduce all dimensions. Only constant value is allowed. Must be in the range [-rank(input_x), rank(input_x)).
- Outputs:
Tensor, has the same dtype as the input_x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the sum of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Raises
TypeError – If keep_dims is not a bool.
TypeError – If input_x is not a Tensor.
ValueError – If axis is None.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = ops.ReduceSum(keep_dims=True) >>> output = op(input_x, 1) >>> output.shape (3, 1, 5, 6)
-
class
tinyms.primitives.Reshape(*args, **kwargs)[source]¶ Reshapes the input tensor with the same values based on a given shape tuple.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
input_shape (tuple[int]) - The input tuple is constructed by multiple integers, i.e., \((y_1, y_2, ..., y_S)\). Only constant value is allowed.
- Outputs:
Tensor, the shape of tensor is \((y_1, y_2, ..., y_S)\).
- Raises
ValueError – Given a shape tuple, if it has several -1; or if the product of its elements is less than or equal to 0 or cannot be divided by the product of the input tensor shape; or if it does not match the input’s array size.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]), mindspore.float32) >>> reshape = ops.Reshape() >>> output = reshape(input_tensor, (3, 2)) >>> print(output) [[-0.1 0.3] [ 3.6 0.4] [ 0.5 -3.2]]
-
class
tinyms.primitives.ResizeBilinear(*args, **kwargs)[source]¶ Resizes an image to a certain size using the bilinear interpolation.
The resizing only affects the lower two dimensions which represent the height and width. The input images can be represented by different data types, but the data types of output images are always float32.
- Parameters
size (Union[tuple[int], list[int]]) – A tuple or list of 2 int elements (new_height, new_width), the new size of the images.
align_corners (bool) – If true, rescale input by (new_height - 1) / (height - 1), which exactly aligns the 4 corners of images and resized images. If false, rescale by new_height / height. Default: False.
- Inputs:
input (Tensor) - Image to be resized. Input images must be a 4-D tensor with shape \((batch, channels, height, width)\), with data type of float32 or float16.
- Outputs:
Tensor, resized image. 4-D with shape [batch, channels, new_height, new_width] in float32.
- Raises
TypeError – If size is neither a tuple nor list.
TypeError – If align_corners is not a bool.
TypeError – If dtype of input is neither float16 nor float32.
TypeError – If input is not a Tensor.
ValueError – If length of shape of input is not equal to 4.
- Supported Platforms:
AscendCPU
Examples
>>> tensor = Tensor([[[[1, 2, 3, 4, 5], [1, 2, 3, 4, 5]]]], mindspore.float32) >>> resize_bilinear = ops.ResizeBilinear((5, 5)) >>> output = resize_bilinear(tensor) >>> print(output) [[[[1. 2. 3. 4. 5.] [1. 2. 3. 4. 5.] [1. 2. 3. 4. 5.] [1. 2. 3. 4. 5.] [1. 2. 3. 4. 5.]]]]
-
class
tinyms.primitives.ResizeNearestNeighbor(*args, **kwargs)[source]¶ Resizes the input tensor by using the nearest neighbor algorithm.
Resizes the input tensor to a given size by using the nearest neighbor algorithm. The nearest neighbor algorithm selects the value of the nearest point and does not consider the values of neighboring points at all, yielding a piecewise-constant interpolant.
- Parameters
- Inputs:
input_x (Tensor) - The input tensor. The shape of the tensor is \((N, C, H, W)\).
- Outputs:
Tensor, the shape of the output tensor is \((N, C, NEW\_H, NEW\_W)\).
- Raises
TypeError – If size is neither tuple nor list.
TypeError – If align_corners is not a bool.
ValueError – If length of size is not equal to 2.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[[[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]]]), mindspore.float32) >>> resize = ops.ResizeNearestNeighbor((2, 2)) >>> output = resize(input_tensor) >>> print(output) [[[[-0.1 0.3] [ 0.4 0.5]]]]
-
class
tinyms.primitives.ReverseSequence(*args, **kwargs)[source]¶ Reverses variable length slices.
- Parameters
- Inputs:
x (Tensor) - The input to reverse, supporting all number types including bool.
seq_lengths (Tensor) - Must be a 1-D vector with int32 or int64 types.
- Outputs:
Reversed tensor with the same shape and data type as input.
- Raises
TypeError – If seq_dim or batch_dim is not an int.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), mindspore.float32) >>> seq_lengths = Tensor(np.array([1, 2, 3])) >>> reverse_sequence = ops.ReverseSequence(seq_dim=1) >>> output = reverse_sequence(x, seq_lengths) >>> print(output) [[1. 2. 3.] [5. 4. 6.] [9. 8. 7.]]
-
class
tinyms.primitives.ReverseV2(*args, **kwargs)[source]¶ Reverses specific dimensions of a tensor.
- Inputs:
input_x (Tensor) - The target tensor.
- Outputs:
Tensor, has the same shape and type as input_x.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[1, 2, 3, 4], [5, 6, 7, 8]]), mindspore.int32) >>> op = ops.ReverseV2(axis=[1]) >>> output = op(input_x) >>> print(output) [[4 3 2 1] [8 7 6 5]]
-
class
tinyms.primitives.Rint(*args, **kwargs)[source]¶ Returns an integer that is closest to x element-wise.
- Inputs:
input_x (Tensor) - The target tensor, which must be one of the following types: float16, float32.
- Outputs:
Tensor, has the same shape and type as input_x.
- Raises
TypeError – If dtype of input_x is neither float16 nor float32.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([-1.6, -0.1, 1.5, 2.0]), mindspore.float32) >>> op = ops.Rint() >>> output = op(input_x) >>> print(output) [-2. 0. 2. 2.]
-
class
tinyms.primitives.Round(*args, **kwargs)[source]¶ Returns half to even of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape and type as the input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([0.8, 1.5, 2.3, 2.5, -4.5]), mindspore.float32) >>> round = ops.Round() >>> output = round(input_x) >>> print(output) [ 1. 2. 2. 2. -4.]
-
class
tinyms.primitives.Rsqrt(*args, **kwargs)[source]¶ Computes reciprocal of square root of input tensor element-wise.
- Inputs:
input_x (Tensor) - The input of Rsqrt. Each element must be a non-negative number.
- Outputs:
Tensor, has the same type and shape as input_x.
- Raises
TypeError – If dtype of input_x is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> input_tensor = Tensor([[4, 4], [9, 9]], mindspore.float32) >>> rsqrt = ops.Rsqrt() >>> output = rsqrt(input_tensor) >>> print(output) [[0.5 0.5 ] [0.33333334 0.33333334]]
-
class
tinyms.primitives.SGD(*args, **kwargs)[source]¶ Computes the stochastic gradient descent. Momentum is optional.
Nesterov momentum is based on the formula from paper On the importance of initialization and momentum in deep learning.
Note
For details, please refer to nn.SGD source code.
- Parameters
- Inputs:
parameters (Tensor) - Parameters to be updated. With float16 or float32 data type.
gradient (Tensor) - Gradient, with float16 or float32 data type.
learning_rate (Tensor) - Learning rate, a scalar tensor with float16 or float32 data type. e.g. Tensor(0.1, mindspore.float32)
accum (Tensor) - Accum(velocity) to be updated. With float16 or float32 data type.
momentum (Tensor) - Momentum, a scalar tensor with float16 or float32 data type. e.g. Tensor(0.1, mindspore.float32).
stat (Tensor) - States to be updated with the same shape as gradient, with float16 or float32 data type.
- Outputs:
Tensor, parameters to be updated.
- Raises
TypeError – If dampening or weight_decay is not a float.
TypeError – If nesterov is not a bool.
TypeError – If parameters, gradient, learning_rate, accum, momentum or stat is not a Tensor.
TypeError – If dtype of parameters, gradient, learning_rate, accum, momentum or stat is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> sgd = ops.SGD() >>> parameters = Tensor(np.array([2, -0.5, 1.7, 4]), mindspore.float32) >>> gradient = Tensor(np.array([1, -1, 0.5, 2]), mindspore.float32) >>> learning_rate = Tensor(0.01, mindspore.float32) >>> accum = Tensor(np.array([0.1, 0.3, -0.2, -0.1]), mindspore.float32) >>> momentum = Tensor(0.1, mindspore.float32) >>> stat = Tensor(np.array([1.5, -0.3, 0.2, -0.7]), mindspore.float32) >>> output = sgd(parameters, gradient, learning_rate, accum, momentum, stat) >>> print(output) (Tensor(shape=[4], dtype=Float32, value= [ 1.98989999e+00, -4.90300000e-01, 1.69520009e+00, 3.98009992e+00]),)
-
class
tinyms.primitives.SameTypeShape(*args, **kwargs)[source]¶ Checks whether the data type and shape of two tensors are the same.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
input_y (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_S)\).
- Outputs:
Tensor, the shape of tensor is \((x_1, x_2, ..., x_R)\), if data type and shape of input_x and input_y are the same.
- Raises
TypeError – If the data types of input_x and input_y are not the same.
ValueError – If the shapes of input_x and input_y are not the same.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> input_y = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> output = ops.SameTypeShape()(input_x, input_y) >>> print(output) [[2. 2.] [2. 2.]]
-
class
tinyms.primitives.ScalarCast(*args, **kwargs)[source]¶ Casts the input scalar to another type.
- Inputs:
input_x (scalar) - The input scalar. Only constant value is allowed.
input_y (mindspore.dtype) - The type to be cast. Only constant value is allowed.
- Outputs:
Scalar. The type is the same as the python type corresponding to input_y.
- Raises
TypeError – If neither input_x nor input_y is a constant value.
- Supported Platforms:
AscendGPUCPU
Examples
>>> scalar_cast = ops.ScalarCast() >>> output = scalar_cast(255.0, mindspore.int32) >>> print(output) 255
-
class
tinyms.primitives.ScalarSummary(*args, **kwargs)[source]¶ Outputs a scalar to a protocol buffer through a scalar summary operator.
- Inputs:
name (str) - The name of the input variable, it must not be an empty string.
value (Tensor) - The value of scalar, and the shape of value must be [] or [1].
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.nn as nn >>> import mindspore.ops as ops >>> >>> >>> class SummaryDemo(nn.Cell): ... def __init__(self,): ... super(SummaryDemo, self).__init__() ... self.summary = ops.ScalarSummary() ... self.add = ops.Add() ... ... def construct(self, x, y): ... name = "x" ... self.summary(name, x) ... x = self.add(x, y) ... return x ...
-
class
tinyms.primitives.ScalarToArray(*args, **kwargs)[source]¶ Converts a scalar to a Tensor.
- Inputs:
input_x (Union[int, float]) - The input is a scalar. Only constant value is allowed.
- Outputs:
Tensor. 0-D Tensor and the content is the input.
- Raises
TypeError – If input_x is neither int nor float.
- Supported Platforms:
AscendGPUCPU
Examples
>>> op = ops.ScalarToArray() >>> data = 1.0 >>> output = op(data) >>> print(output) 1.0
-
class
tinyms.primitives.ScalarToTensor(*args, **kwargs)[source]¶ Converts a scalar to a Tensor, and converts the data type to the specified type.
- Inputs:
input_x (Union[int, float]) - The input is a scalar. Only constant value is allowed.
dtype (mindspore.dtype) - The target data type. Default: mindspore.float32. Only constant value is allowed.
- Outputs:
Tensor. 0-D Tensor and the content is the input.
- Raises
TypeError – If input_x is neither int nor float.
- Supported Platforms:
AscendGPUCPU
Examples
>>> op = ops.ScalarToTensor() >>> data = 1 >>> output = op(data, mindspore.float32) >>> print(output) 1.0
-
class
tinyms.primitives.ScatterAdd(*args, **kwargs)[source]¶ Updates the value of the input tensor through the addition operation.
Using given values to update tensor value through the add operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
for each i, …, j in indices.shape:
\[\text{input_x}[\text{indices}[i, ..., j], :] \mathrel{+}= \text{updates}[i, ..., j, :]\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do add operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the add operation with input_x, the data type is the same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Parameter(Tensor(np.array([[0.0, 0.0, 0.0], [0.0, 0.0, 0.0]]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[0, 1], [1, 1]]), mindspore.int32) >>> updates = Tensor(np.ones([2, 2, 3]), mindspore.float32) >>> scatter_add = ops.ScatterAdd() >>> output = scatter_add(input_x, indices, updates) >>> print(output) [[1. 1. 1.] [3. 3. 3.]]
-
class
tinyms.primitives.ScatterDiv(*args, **kwargs)[source]¶ Updates the value of the input tensor through the divide operation.
Using given values to update tensor value through the div operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
for each i, …, j in indices.shape:
\[\text{input_x}[\text{indices}[i, ..., j], :] \mathrel{/}= \text{updates}[i, ..., j, :]\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do div operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the div operation with input_x, the data type is the same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[6.0, 6.0, 6.0], [2.0, 2.0, 2.0]]), mindspore.float32), name="x") >>> indices = Tensor(np.array([0, 1]), mindspore.int32) >>> updates = Tensor(np.array([[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]), mindspore.float32) >>> scatter_div = ops.ScatterDiv() >>> output = scatter_div(input_x, indices, updates) >>> print(output) [[3. 3. 3.] [1. 1. 1.]]
-
class
tinyms.primitives.ScatterMax(*args, **kwargs)[source]¶ Updates the value of the input tensor through the maximum operation.
Using given values to update tensor value through the max operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
for each i, …, j in indices.shape:
\[\text{input_x}[\text{indices}[i, ..., j], :] = max(\text{input_x}[\text{indices}[i, ..., j], :], \text{updates}[i, ..., j, :])\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: True.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do max operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the maximum operation with input_x, the data type is the same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]), mindspore.float32), name="input_x") >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> updates = Tensor(np.ones([2, 2, 3]) * 88, mindspore.float32) >>> scatter_max = ops.ScatterMax() >>> output = scatter_max(input_x, indices, updates) >>> print(output) [[88. 88. 88.] [88. 88. 88.]]
-
class
tinyms.primitives.ScatterMin(*args, **kwargs)[source]¶ Updates the value of the input tensor through the minimum operation.
Using given values to update tensor value through the min operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
for each i, …, j in indices.shape:
\[\text{input_x}[\text{indices}[i, ..., j], :] = min(\text{input_x}[\text{indices}[i, ..., j], :], \text{updates}[i, ..., j, :])\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do min operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor doing the min operation with input_x, the data type is same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[0.0, 1.0, 2.0], [0.0, 0.0, 0.0]]), mindspore.float32), name="input_x") >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> update = Tensor(np.ones([2, 2, 3]), mindspore.float32) >>> scatter_min = ops.ScatterMin() >>> output = scatter_min(input_x, indices, update) >>> print(output) [[0. 1. 1.] [0. 0. 0.]]
-
class
tinyms.primitives.ScatterMul(*args, **kwargs)[source]¶ Updates the value of the input tensor through the multiply operation.
Using given values to update tensor value through the mul operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
for each i, …, j in indices.shape:
\[\text{input_x}[\text{indices}[i, ..., j], :] \mathrel{*}= \text{updates}[i, ..., j, :]\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do mul operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor doing the mul operation with input_x, the data type is same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[1.0, 1.0, 1.0], [2.0, 2.0, 2.0]]), mindspore.float32), name="x") >>> indices = Tensor(np.array([0, 1]), mindspore.int32) >>> updates = Tensor(np.array([[2.0, 2.0, 2.0], [2.0, 2.0, 2.0]]), mindspore.float32) >>> scatter_mul = ops.ScatterMul() >>> output = scatter_mul(input_x, indices, updates) >>> print(output) [[2. 2. 2.] [4. 4. 4.]]
-
class
tinyms.primitives.ScatterNd(*args, **kwargs)[source]¶ Scatters a tensor into a new tensor depending on the specified indices.
Creates an empty tensor with the given shape, and set values by scattering the update tensor depending on indices.
The empty tensor has rank P and indices has rank Q where Q >= 2.
indices has shape \((i_0, i_1, ..., i_{Q-2}, N)\) where N <= P.
The last dimension of indices (with length N ) indicates slices along the N th dimension of the empty tensor.
updates is a tensor of rank Q-1+P-N. Its shape is: \((i_0, i_1, ..., i_{Q-2}, shape_N, ..., shape_{P-1})\).
- Inputs:
indices (Tensor) - The index of scattering in the new tensor with int32 data type. The rank of indices must be at least 2 and indices_shape[-1] <= len(shape).
updates (Tensor) - The source Tensor to be scattered. It has shape indices_shape[:-1] + shape[indices_shape[-1]:].
shape (tuple[int]) - Define the shape of the output tensor, has the same type as indices.
- Outputs:
Tensor, the new tensor, has the same type as update and the same shape as shape.
- Raises
TypeError – If shape is not a tuple.
ValueError – If any element of shape is less than 1.
- Supported Platforms:
AscendGPU
Examples
>>> op = ops.ScatterNd() >>> indices = Tensor(np.array([[0, 1], [1, 1]]), mindspore.int32) >>> updates = Tensor(np.array([3.2, 1.1]), mindspore.float32) >>> shape = (3, 3) >>> output = op(indices, updates, shape) >>> print(output) [[0. 3.2 0. ] [0. 1.1 0. ] [0. 0. 0. ]]
-
class
tinyms.primitives.ScatterNdAdd(*args, **kwargs)[source]¶ Applies sparse addition to individual values or slices in a tensor.
Using given values to update tensor value through the add operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
input_x has rank P and indices has rank Q where Q >= 2.
indices has shape \((i_0, i_1, ..., i_{Q-2}, N)\) where N <= P.
The last dimension of indices (with length N ) indicates slices along the N th dimension of input_x.
updates is a tensor of rank Q-1+P-N. Its shape is: \((i_0, i_1, ..., i_{Q-2}, x\_shape_N, ..., x\_shape_{P-1})\).
Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do add operation whose data type must be mindspore.int32. The rank of indices must be at least 2 and indices_shape[-1] <= len(shape).
updates (Tensor) - The tensor doing the add operation with input_x, the data type is same as input_x, the shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Outputs:
Parameter, the updated input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([1, 2, 3, 4, 5, 6, 7, 8]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[2], [4], [1], [7]]), mindspore.int32) >>> updates = Tensor(np.array([6, 7, 8, 9]), mindspore.float32) >>> scatter_nd_add = ops.ScatterNdAdd() >>> output = scatter_nd_add(input_x, indices, updates) >>> print(output) [ 1. 10. 9. 4. 12. 6. 7. 17.]
-
class
tinyms.primitives.ScatterNdSub(*args, **kwargs)[source]¶ Applies sparse subtraction to individual values or slices in a tensor.
Using given values to update tensor value through the subtraction operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
input_x has rank P and indices has rank Q where Q >= 2.
indices has shape \((i_0, i_1, ..., i_{Q-2}, N)\) where N <= P.
The last dimension of indices (with length N ) indicates slices along the N th dimension of input_x.
updates is a tensor of rank Q-1+P-N. Its shape is: \((i_0, i_1, ..., i_{Q-2}, x\_shape_N, ..., x\_shape_{P-1})\).
Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to do add operation whose data type must be mindspore.int32. The rank of indices must be at least 2 and indices_shape[-1] <= len(shape).
updates (Tensor) - The tensor that performs the subtraction operation with input_x, the data type is the same as input_x, the shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Outputs:
Parameter, the updated input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([1, 2, 3, 4, 5, 6, 7, 8]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[2], [4], [1], [7]]), mindspore.int32) >>> updates = Tensor(np.array([6, 7, 8, 9]), mindspore.float32) >>> scatter_nd_sub = ops.ScatterNdSub() >>> output = scatter_nd_sub(input_x, indices, updates) >>> print(output) [ 1. -6. -3. 4. -2. 6. 7. -1.]
-
class
tinyms.primitives.ScatterNdUpdate(*args, **kwargs)[source]¶ Updates tensor values by using input indices and value.
Using given values to update tensor value, along with the input indices.
input_x has rank P and indices has rank Q where Q >= 2.
indices has shape \((i_0, i_1, ..., i_{Q-2}, N)\) where N <= P.
The last dimension of indices (with length N ) indicates slices along the N th dimension of input_x.
updates is a tensor of rank Q-1+P-N. Its shape is: \((i_0, i_1, ..., i_{Q-2}, x\_shape_N, ..., x\_shape_{P-1})\).
Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: True.
- Inputs:
input_x (Parameter) - The target tensor, with data type of Parameter.
indices (Tensor) - The index of input tensor, with int32 data type. The rank of indices must be at least 2 and indices_shape[-1] <= len(shape).
updates (Tensor) - The tensor to be updated to the input tensor, has the same type as input. The shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Outputs:
Tensor, has the same shape and type as input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
AscendCPU
Examples
>>> np_x = np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]) >>> input_x = mindspore.Parameter(Tensor(np_x, mindspore.float32), name="x") >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> updates = Tensor(np.array([1.0, 2.2]), mindspore.float32) >>> op = ops.ScatterNdUpdate() >>> output = op(input_x, indices, updates) >>> print(output) [[1. 0.3 3.6] [0.4 2.2 -3.2]]
-
class
tinyms.primitives.ScatterNonAliasingAdd(*args, **kwargs)[source]¶ Applies sparse addition to the input using individual values or slices.
Using given values to update tensor value through the add operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Inputs:
input_x (Parameter) - The target parameter. The data type must be float16, float32 or int32.
indices (Tensor) - The index to perform the addition operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the addition operation with input_x, the data type is the same as input_x, the shape is indices_shape[:-1] + x_shape[indices_shape[-1]:].
- Outputs:
Parameter, the updated input_x.
- Raises
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([1, 2, 3, 4, 5, 6, 7, 8]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[2], [4], [1], [7]]), mindspore.int32) >>> updates = Tensor(np.array([6, 7, 8, 9]), mindspore.float32) >>> scatter_non_aliasing_add = ops.ScatterNonAliasingAdd() >>> output = scatter_non_aliasing_add(input_x, indices, updates) >>> print(output) [ 1. 10. 9. 4. 12. 6. 7. 17.]
-
class
tinyms.primitives.ScatterSub(*args, **kwargs)[source]¶ Updates the value of the input tensor through the subtraction operation.
Using given values to update tensor value through the subtraction operation, along with the input indices. This operation outputs the input_x after the update is done, which makes it convenient to use the updated value.
for each i, …, j in indices.shape:
\[\text{input_x}[\text{indices}[i, ..., j], :] \mathrel{-}= \text{updates}[i, ..., j, :]\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: False.
- Inputs:
input_x (Parameter) - The target parameter.
indices (Tensor) - The index to perform the subtraction operation whose data type must be mindspore.int32.
updates (Tensor) - The tensor that performs the subtraction operation with input_x, the data type is the same as input_x, the shape is indices_shape + x_shape[1:].
- Outputs:
Parameter, the updated input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Parameter(Tensor(np.array([[0.0, 0.0, 0.0], [1.0, 1.0, 1.0]]), mindspore.float32), name="x") >>> indices = Tensor(np.array([[0, 1]]), mindspore.int32) >>> updates = Tensor(np.array([[[1.0, 1.0, 1.0], [2.0, 2.0, 2.0]]]), mindspore.float32) >>> scatter_sub = ops.ScatterSub() >>> output = scatter_sub(input_x, indices, updates) >>> print(output) [[-1. -1. -1.] [-1. -1. -1.]]
-
class
tinyms.primitives.ScatterUpdate(*args, **kwargs)[source]¶ Updates tensor values by using input indices and value.
Using given values to update tensor value, along with the input indices.
for each i, …, j in indices.shape:
\[\text{input_x}[\text{indices}[i, ..., j], :] = \text{updates}[i, ..., j, :]\]Inputs of input_x and updates comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – Whether protect the assignment by a lock. Default: True.
- Inputs:
input_x (Parameter) - The target tensor, with data type of Parameter.
indices (Tensor) - The index of input tensor. With int32 data type. If there are duplicates in indices, the order for updating is undefined.
updates (Tensor) - The tensor to update the input tensor, has the same type as input, and updates.shape = indices.shape + input_x.shape[1:].
- Outputs:
Tensor, has the same shape and type as input_x.
- Raises
TypeError – If use_locking is not a bool.
- Supported Platforms:
AscendGPU
Examples
>>> np_x = np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]) >>> input_x = mindspore.Parameter(Tensor(np_x, mindspore.float32), name="x") >>> indices = Tensor(np.array([0, 1]), mindspore.int32) >>> np_updates = np.array([[2.0, 1.2, 1.0], [3.0, 1.2, 1.0]]) >>> updates = Tensor(np_updates, mindspore.float32) >>> op = ops.ScatterUpdate() >>> output = op(input_x, indices, updates) >>> print(output) [[2. 1.2 1. ] [3. 1.2 1. ]]
-
class
tinyms.primitives.SeLU(*args, **kwargs)[source]¶ Computes SeLU (scaled exponential Linear Unit) of input tensors element-wise.
The activation function is defined as:
\[E_{i} = scale * \begin{cases} x, &\text{if } x \geq 0; \cr \text{alpha} * (\exp(x_i) - 1), &\text{otherwise.} \end{cases}\]where \(alpha\) and \(scale\) are pre-defined constants(\(alpha=1.67326324\) and \(scale=1.05070098\)).
See more details in Self-Normalizing Neural Networks.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Supported Platforms:
Ascend- Raise:
TypeError: If dtype of input_x is neither float16 nor float32.
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> selu = ops.SeLU() >>> output = selu(input_x) >>> print(output) [[-1.1113307 4.202804 -1.7575096] [ 2.101402 -1.7462534 9.456309 ]]
-
class
tinyms.primitives.Select(*args, **kwargs)[source]¶ Returns the selected elements, either from input \(x\) or input \(y\), depending on the condition.
Given a tensor as input, this operation inserts a dimension of 1 at the dimension, it was invalid when both math: ‘x’ and math: ‘y’ are none. Keep in mind that the shape of the output tensor can vary depending on how many true values are in the input. Indexes are output in row-first order.
If neither is None, math:x and \(y\) must have the same shape. If \(x\) and \(y\) are scalars, the conditional tensor must be a scalar. If \(x\) and \(y\) are higher-dimensional vectors, the condition must be a vector whose size matches the first dimension of \(x\), or must have the same shape as \(y\).
The conditional tensor acts as an optional compensation (mask), which determines whether the corresponding element / row in the output must be selected from \(x\) (if true) or \(y\) (if false) based on the value of each element.
It can be defined as:
\[\begin{split}out_i = \begin{cases} x_i, & \text{if } condition_i \\ y_i, & \text{otherwise} \end{cases}\end{split}\]If condition is a vector, then \(x\) and \(y\) are higher-dimensional matrices, then it chooses to copy that row (external dimensions) from \(x\) and \(y\). If condition has the same shape as \(x\) and \(y\), you can choose to copy these elements from \(x\) and \(y\).
- Inputs:
input_cond (Tensor[bool]) - The shape is \((x_1, x_2, ..., x_N, ..., x_R)\). The condition tensor, decides which element is chosen.
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_N, ..., x_R)\). The first input tensor.
input_y (Tensor) - The shape is \((x_1, x_2, ..., x_N, ..., x_R)\). The second input tensor.
- Outputs:
Tensor, has the same shape as input_x. The shape is \((x_1, x_2, ..., x_N, ..., x_R)\).
- Raises
TypeError – If input_x or input_y is not a Tensor.
ValueError – If shape of input_x is not equal to shape of input_y or shape of input_cond.
- Supported Platforms:
AscendGPUCPU
Examples
>>> select = ops.Select() >>> input_cond = Tensor([True, False]) >>> input_x = Tensor([2,3], mindspore.float32) >>> input_y = Tensor([1,2], mindspore.float32) >>> output = select(input_cond, input_x, input_y) >>> print(output) [2. 2.]
-
class
tinyms.primitives.Shape(*args, **kwargs)[source]¶ Returns the shape of the input tensor.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
tuple[int], the output tuple is constructed by multiple integers, \((x_1, x_2, ..., x_R)\).
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) >>> shape = ops.Shape() >>> output = shape(input_tensor) >>> print(output) (3, 2, 1)
-
class
tinyms.primitives.Sigmoid(*args, **kwargs)[source]¶ Sigmoid activation function.
Computes Sigmoid of input element-wise. The Sigmoid function is defined as:
\[\text{sigmoid}(x_i) = \frac{1}{1 + \exp(-x_i)},\]where \(x_i\) is the element of the input.
- Inputs:
input_x (Tensor) - The input of Sigmoid, data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> sigmoid = ops.Sigmoid() >>> output = sigmoid(input_x) >>> print(output) [0.7310586 0.880797 0.95257413 0.98201376 0.9933072 ]
-
class
tinyms.primitives.SigmoidCrossEntropyWithLogits(*args, **kwargs)[source]¶ Uses the given logits to compute sigmoid cross entropy between the target and the output.
Measures the distribution error in discrete classification tasks where each class is independent and not mutually exclusive using cross entropy loss.
Sets input logits as X, input label as Y, output as loss. Then,
\[p_{ij} = sigmoid(X_{ij}) = \frac{1}{1 + e^{-X_{ij}}}\]\[loss_{ij} = -[Y_{ij} * ln(p_{ij}) + (1 - Y_{ij})ln(1 - p_{ij})]\]- Inputs:
logits (Tensor) - Input logits.
label (Tensor) - Ground truth label. With the same shape and type as logits.
- Outputs:
Tensor, with the same shape and type as input logits.
- Raises
TypeError – If logits or label is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> logits = Tensor(np.array([[-0.8, 1.2, 0.7], [-0.1, -0.4, 0.7]]).astype(np.float32)) >>> labels = Tensor(np.array([[0.3, 0.8, 1.2], [-0.6, 0.1, 2.2]]).astype(np.float32)) >>> sigmoid = ops.SigmoidCrossEntropyWithLogits() >>> output = sigmoid(logits, labels) >>> print(output) [[ 0.6111007 0.5032824 0.26318604] [ 0.58439666 0.5530153 -0.4368139 ]]
-
class
tinyms.primitives.Sign(*args, **kwargs)[source]¶ Performs sign on the tensor element-wise.
\[sign(x) = \begin{cases} -1, &if\ x < 0 \cr 0, &if\ x = 0 \cr 1, &if\ x > 0\end{cases}\]- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, has the same shape and type as the input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendCPU
Examples
>>> input_x = Tensor(np.array([[2.0, 0.0, -1.0]]), mindspore.float32) >>> sign = ops.Sign() >>> output = sign(input_x) >>> print(output) [[ 1. 0. -1.]]
-
class
tinyms.primitives.Sin(*args, **kwargs)[source]¶ Computes sine of the input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> sin = ops.Sin() >>> input_x = Tensor(np.array([0.62, 0.28, 0.43, 0.62]), mindspore.float32) >>> output = sin(input_x) >>> print(output) [0.5810352 0.27635565 0.41687083 0.5810352 ]
-
class
tinyms.primitives.Sinh(*args, **kwargs)[source]¶ Computes hyperbolic sine of the input element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, has the same shape as input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendCPU
Examples
>>> sinh = ops.Sinh() >>> input_x = Tensor(np.array([0.62, 0.28, 0.43, 0.62]), mindspore.float32) >>> output = sinh(input_x) >>> print(output) [0.6604918 0.28367308 0.44337422 0.6604918 ]
-
class
tinyms.primitives.Size(*args, **kwargs)[source]¶ Returns the size of a tensor.
Returns an int scalar representing the elements size of input, the total number of elements in the tensor.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
int, a scalar representing the elements size of input_x, tensor is the number of elements in a tensor, \(size=x_1*x_2*...x_R\).
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[2, 2], [2, 2]]), mindspore.float32) >>> size = ops.Size() >>> output = size(input_tensor) >>> print(output) 4
-
class
tinyms.primitives.Slice(*args, **kwargs)[source]¶ Slices a tensor in the specified shape.
Slice the tensor ‘input_x` in shape of size and starting at the location specified by begin, The slice begin represents the offset in each dimension of input_x, The slice size represents the size of the output tensor.
Note that begin is zero-based and size is one-based.
If size[i] is -1, all remaining elements in dimension i are included in the slice. This is equivalent to setting \(size[i] = input_x.shape(i) - begin[i]\)
- Inputs:
input_x (Tensor): The target tensor.
begin (Union[tuple, list]): The beginning of the slice. Only constant value is allowed.
size (Union[tuple, list]): The size of the slice. Only constant value is allowed.
- Outputs:
Tensor, the shape is : input size, the data type is the same as input_x.
- Raises
TypeError – If begin or size is neither tuple nor list.
- Supported Platforms:
AscendGPUCPU
Examples
>>> data = Tensor(np.array([[[1, 1, 1], [2, 2, 2]], ... [[3, 3, 3], [4, 4, 4]], ... [[5, 5, 5], [6, 6, 6]]]).astype(np.int32)) >>> slice = ops.Slice() >>> output = slice(data, (1, 0, 0), (1, 1, 3)) >>> print(output) [[[3 3 3]]]
-
class
tinyms.primitives.SmoothL1Loss(*args, **kwargs)[source]¶ Computes smooth L1 loss, a robust L1 loss.
SmoothL1Loss is a Loss similar to MSELoss but less sensitive to outliers as described in the Fast R-CNN by Ross Girshick.
The updating formulas of SmoothL1Loss algorithm are as follows,
\[\text{SmoothL1Loss} = \begin{cases} \frac{0.5 x^{2}}{\text{beta}}, &if \left |x \right | < \text{beta} \cr \left |x \right|-0.5 \text{beta}, &\text{otherwise}\end{cases}\]where \(X\) represents prediction. \(Y\) represents target. \(loss\) represents output.
- Parameters
beta (float) – A parameter used to control the point where the function will change from quadratic to linear. Default: 1.0.
- Inputs:
prediction (Tensor) - Predict data. Data type must be float16 or float32.
target (Tensor) - Ground truth data, with the same type and shape as prediction.
- Outputs:
Tensor, with the same type and shape as prediction.
- Raises
TypeError – If beta is not a float.
TypeError – If prediction or target is not a Tensor.
TypeError – If dtype of prediction or target is neither float16 nor float32.
ValueError – If beta is less than or equal to 0.
ValueError – If shape of prediction is not the same as target.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = ops.SmoothL1Loss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) [0. 0. 0.5]
-
class
tinyms.primitives.Softmax(*args, **kwargs)[source]¶ Softmax operation.
Applies the Softmax operation to the input tensor on the specified axis. Suppose a slice in the given aixs \(x\), then for each element \(x_i\), the Softmax function is shown as follows:
\[\text{output}(x_i) = \frac{exp(x_i)}{\sum_{j = 0}^{N-1}\exp(x_j)},\]where \(N\) is the length of the tensor.
- Inputs:
logits (Tensor) - The input of Softmax, with float16 or float32 data type.
- Outputs:
Tensor, with the same type and shape as the logits.
- Raises
TypeError – If axis is neither an int nor a tuple.
TypeError – If dtype of logits is neither float16 nor float32.
ValueError – If axis is a tuple whose length is less than 1.
ValueError – If axis is a tuple whose elements are not all in range [-len(logits), len(logits)).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> softmax = ops.Softmax() >>> output = softmax(input_x) >>> print(output) [0.01165623 0.03168492 0.08612854 0.23412167 0.6364086 ]
-
class
tinyms.primitives.SoftmaxCrossEntropyWithLogits(*args, **kwargs)[source]¶ Gets the softmax cross-entropy value between logits and labels with one-hot encoding.
The updating formulas of SoftmaxCrossEntropyWithLogits algorithm are as follows,
\[\begin{split}\begin{array}{ll} \\ p_{ij} = softmax(X_{ij}) = \frac{\exp(x_i)}{\sum_{j = 0}^{N-1}\exp(x_j)} \\ loss_{ij} = -\sum_j{Y_{ij} * ln(p_{ij})} \end{array}\end{split}\]where \(X\) represents logits. \(Y\) represents label. \(loss\) represents output.
- Inputs:
logits (Tensor) - Input logits, with shape \((N, C)\). Data type must be float16 or float32.
labels (Tensor) - Ground truth labels, with shape \((N, C)\), has the same data type with logits.
- Outputs:
Tuple of 2 tensors, the loss shape is (N,), and the dlogits with the same shape as logits.
- Raises
TypeError – If dtype of logits or labels is neither float16 nor float32.
TypeError – If logits or labels is not a Tensor.
ValueError – If shape of logits is not the same as labels.
- Supported Platforms:
AscendGPUCPU
Examples
>>> logits = Tensor([[2, 4, 1, 4, 5], [2, 1, 2, 4, 3]], mindspore.float32) >>> labels = Tensor([[0, 0, 0, 0, 1], [0, 0, 0, 1, 0]], mindspore.float32) >>> softmax_cross = ops.SoftmaxCrossEntropyWithLogits() >>> loss, dlogits = softmax_cross(logits, labels) >>> print(loss) [0.5899297 0.52374405] >>> print(dlogits) [[ 0.02760027 0.20393994 0.01015357 0.20393994 -0.44563377] [ 0.08015892 0.02948882 0.08015892 -0.4077012 0.21789455]]
-
class
tinyms.primitives.Softplus(*args, **kwargs)[source]¶ Softplus activation function.
Softplus is a smooth approximation to the ReLU function. It can be used to constrain the output of a machine to always be positive. The function is shown as follows:
\[\text{output} = \log(1 + \exp(\text{input_x})),\]- Inputs:
input_x (Tensor) - The input tensor whose data type must be float.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> softplus = ops.Softplus() >>> output = softplus(input_x) >>> print(output) [1.3132615 2.126928 3.0485873 4.01815 5.0067153]
-
class
tinyms.primitives.Softsign(*args, **kwargs)[source]¶ Softsign activation function.
The function is shown as follows:
\[\text{SoftSign}(x) = \frac{x}{ 1 + |x|}\]- Inputs:
input_x (Tensor) - The input tensor whose data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Raises
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([0, -1, 2, 30, -30]), mindspore.float32) >>> softsign = ops.Softsign() >>> output = softsign(input_x) >>> print(output) [ 0. -0.5 0.6666667 0.9677419 -0.9677419]
-
class
tinyms.primitives.Sort(*args, **kwargs)[source]¶ Sorts the elements of the input tensor along a given dimension in ascending order by value.
- Parameters
- Inputs:
x (Tensor) - The input to sort, with float16 or float32 data type.
- Outputs:
y1 (Tensor) - A tensor whose values are the sorted values, with the same shape and data type as input.
y2 (Tensor) - The indices of the elements in the original input tensor. Data type is int32.
- Raises
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([[8, 2, 1], [5, 9, 3], [4, 6, 7]]), mindspore.float16) >>> sort = ops.Sort() >>> output = sort(x) >>> print(output) (Tensor(shape=[3, 3], dtype=Float16, value= [[ 1.0000e+00, 2.0000e+00, 8.0000e+00], [ 3.0000e+00, 5.0000e+00, 9.0000e+00], [ 4.0000e+00, 6.0000e+00, 7.0000e+00]]), Tensor(shape=[3, 3], dtype=Int32, value= [[2, 1, 0], [2, 0, 1], [0, 1, 2]]))
-
class
tinyms.primitives.SpaceToBatch(*args, **kwargs)[source]¶ Divides spatial dimensions into blocks and combines the block size with the original batch.
This operation will divide spatial dimensions (H, W) into blocks with block_size, the output tensor’s H and W dimension is the corresponding number of blocks after division. The output tensor’s batch dimension is the product of the original batch and the square of block_size. Before division, the spatial dimensions of the input are zero padded according to paddings if necessary.
- Parameters
block_size (int) – The block size of dividing blocks with value greater than or euqual to 2.
paddings (Union[tuple, list]) – The padding values for H and W dimension, containing 2 subtraction lists. Each subtraction list contains 2 integer value. All values must be greater than 0. paddings[i] specifies the paddings for the spatial dimension i, which corresponds to the input dimension i+2. It is required that input_shape[i+2]+paddings[i][0]+paddings[i][1] is divisible by block_size.
- Inputs:
input_x (Tensor) - The input tensor. It must be a 4-D tensor.
- Outputs:
Tensor, the output tensor with the same data type as input. Assume input shape is \((n, c, h, w)\) with \(block\_size\) and \(paddings\). The shape of the output tensor will be \((n', c', h', w')\), where
\(n' = n*(block\_size*block\_size)\)
\(c' = c\)
\(h' = (h+paddings[0][0]+paddings[0][1])//block\_size\)
\(w' = (w+paddings[1][0]+paddings[1][1])//block\_size\)
- Raises
TypeError – If block_size is not an int.
ValueError – If block_size is less than 2.
- Supported Platforms:
Ascend
Examples
>>> block_size = 2 >>> paddings = [[0, 0], [0, 0]] >>> space_to_batch = ops.SpaceToBatch(block_size, paddings) >>> input_x = Tensor(np.array([[[[1, 2], [3, 4]]]]), mindspore.float32) >>> output = space_to_batch(input_x) >>> print(output) [[[[1.]]] [[[2.]]] [[[3.]]] [[[4.]]]]
-
class
tinyms.primitives.SpaceToBatchND(*args, **kwargs)[source]¶ Divides spatial dimensions into blocks and combines the block size with the original batch.
This operation will divide spatial dimensions (H, W) into blocks with block_shape, the output tensor’s H and W dimension is the corresponding number of blocks after division. The output tensor’s batch dimension is the product of the original batch and the product of block_shape. Before division, the spatial dimensions of the input are zero padded according to paddings if necessary.
- Parameters
block_shape (Union[list(int), tuple(int), int]) – The block shape of dividing block with all value greater than 1. If block_shape is a tuple or list, the length of block_shape is M corresponding to the number of spatial dimensions. If block_shape is a int, the block size of M dimendions are the same, equal to block_shape. M must be 2.
paddings (Union[tuple, list]) – The padding values for H and W dimension, containing 2 subtraction list. Each contains 2 integer value. All values must be greater than 0. paddings[i] specifies the paddings for the spatial dimension i, which corresponds to the input dimension i+2. It is required that input_shape[i+2]+paddings[i][0]+paddings[i][1] is divisible by block_shape[i].
- Inputs:
input_x (Tensor) - The input tensor. It must be a 4-D tensor.
- Outputs:
Tensor, the output tensor with the same data type as input. Assume input shape is \((n, c, h, w)\) with \(block\_shape\) and \(padddings\). The shape of the output tensor will be \((n', c', h', w')\), where
\(n' = n*(block\_shape[0]*block\_shape[1])\)
\(c' = c\)
\(h' = (h+paddings[0][0]+paddings[0][1])//block\_shape[0]\)
\(w' = (w+paddings[1][0]+paddings[1][1])//block\_shape[1]\)
- Raises
TypeError – If block_shape is not one of list, tuple, int.
TypeError – If paddings is neither list nor tuple.
ValueError – If length of shape of block_shape is not equal to 1.
ValueError – If length of block_shape or paddings is not equal to 2.
- Supported Platforms:
Ascend
Examples
>>> block_shape = [2, 2] >>> paddings = [[0, 0], [0, 0]] >>> space_to_batch_nd = ops.SpaceToBatchND(block_shape, paddings) >>> input_x = Tensor(np.array([[[[1, 2], [3, 4]]]]), mindspore.float32) >>> output = space_to_batch_nd(input_x) >>> print(output) [[[[1.]]] [[[2.]]] [[[3.]]] [[[4.]]]]
-
class
tinyms.primitives.SpaceToDepth(*args, **kwargs)[source]¶ Rearranges blocks of spatial data into depth.
The output tensor’s height dimension is \(height / block\_size\).
The output tensor’s weight dimension is \(weight / block\_size\).
The depth of output tensor is \(block\_size * block\_size * input\_depth\).
The input tensor’s height and width must be divisible by block_size. The data format is “NCHW”.
- Parameters
block_size (int) – The block size used to divide spatial data. It must be >= 2.
- Inputs:
x (Tensor) - The target tensor.
- Outputs:
Tensor, the same data type as x. It must be a 4-D tensor.
- Raises
TypeError – If block_size is not an int.
ValueError – If block_size is less than 2.
ValueError – If length of shape of x is not equal to 4.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.random.rand(1,3,2,2), mindspore.float32) >>> block_size = 2 >>> space_to_depth = ops.SpaceToDepth(block_size) >>> output = space_to_depth(x) >>> print(output.shape) (1, 12, 1, 1)
-
class
tinyms.primitives.SparseApplyAdagrad(*args, **kwargs)[source]¶ Updates relevant entries according to the adagrad scheme.
\[accum += grad * grad\]\[var -= lr * grad * (1 / sqrt(accum))\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
- Inputs:
var (Parameter) - Variable to be updated. The data type must be float16 or float32.
accum (Parameter) - Accumulation to be updated. The shape and data type must be the same as var.
grad (Tensor) - Gradient. The shape must be the same as var’s shape except the first dimension. Gradients has the same data type as var.
indices (Tensor) - A vector of indices into the first dimension of var and accum. The shape of indices must be the same as grad in first dimension, the type must be int32.
- Outputs:
Tuple of 2 tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_adagrad = ops.SparseApplyAdagrad(lr=1e-8) ... self.var = Parameter(Tensor(np.ones([1, 1, 1]).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.ones([1, 1, 1]).astype(np.float32)), name="accum") ... def construct(self, grad, indices): ... out = self.sparse_apply_adagrad(self.var, self.accum, grad, indices) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(1, 1, 1).astype(np.float32)) >>> indices = Tensor([0], mstype.int32) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1, 1, 1], dtype=Float32, value= [[[1.00000000e+00]]]), Tensor(shape=[1, 1, 1], dtype=Float32, value= [[[1.00000000e+00]]]))
-
class
tinyms.primitives.SparseApplyAdagradV2(*args, **kwargs)[source]¶ Updates relevant entries according to the adagrad scheme, one more epsilon attribute than SparseApplyAdagrad.
\[accum += grad * grad\]\[var -= lr * grad * \frac{1}{\sqrt{accum} + \epsilon}\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
- Inputs:
var (Parameter) - Variable to be updated. The data type must be float16 or float32.
accum (Parameter) - Accumulation to be updated. The shape and data type must be the same as var.
grad (Tensor) - Gradient. The shape must be the same as var’s shape except the first dimension. Gradients has the same data type as var.
indices (Tensor) - A vector of indices into the first dimension of var and accum. The shape of indices must be the same as grad in first dimension, the type must be int32.
- Outputs:
Tuple of 2 tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> import mindspore.common.dtype as mstype >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_adagrad_v2 = ops.SparseApplyAdagradV2(lr=1e-8, epsilon=1e-6) ... self.var = Parameter(Tensor(np.ones([1, 1, 1]).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.ones([1, 1, 1]).astype(np.float32)), name="accum") ... ... def construct(self, grad, indices): ... out = self.sparse_apply_adagrad_v2(self.var, self.accum, grad, indices) ... return out ... >>> np.random.seed(0) >>> net = Net() >>> grad = Tensor(np.random.rand(1, 1, 1).astype(np.float32)) >>> indices = Tensor([0], mstype.int32) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1, 1, 1], dtype=Float32, value= [[[1.00000000e+00]]]), Tensor(shape=[1, 1, 1], dtype=Float32, value= [[[1.30119634e+00]]]))
-
class
tinyms.primitives.SparseApplyFtrl(*args, **kwargs)[source]¶ Updates relevant entries according to the FTRL-proximal scheme.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
lr (float) – The learning rate value, must be positive.
l1 (float) – l1 regularization strength, must be greater than or equal to zero.
l2 (float) – l2 regularization strength, must be greater than or equal to zero.
lr_power (float) – Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero.
use_locking (bool) – Use locks for updating operation if true . Default: False.
- Inputs:
var (Parameter) - The variable to be updated. The data type must be float16 or float32.
accum (Parameter) - The accumulation to be updated, must be same data type and shape as var.
linear (Parameter) - the linear coefficient to be updated, must be the same data type and shape as var.
grad (Tensor) - A tensor of the same type as var, for the gradient.
indices (Tensor) - A tensor of indices in the first dimension of var and accum. The shape of indices must be the same as grad in the first dimension. If there are duplicates in indices, the behavior is undefined. The type must be int32 or int64.
- Outputs:
var (Tensor) - Tensor, has the same shape and data type as var.
accum (Tensor) - Tensor, has the same shape and data type as accum.
linear (Tensor) - Tensor, has the same shape and data type as linear.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class SparseApplyFtrlNet(nn.Cell): ... def __init__(self): ... super(SparseApplyFtrlNet, self).__init__() ... self.sparse_apply_ftrl = ops.SparseApplyFtrl(lr=0.01, l1=0.0, l2=0.0, lr_power=-0.5) ... self.var = Parameter(Tensor(np.random.rand(1, 1).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(1, 1).astype(np.float32)), name="accum") ... self.linear = Parameter(Tensor(np.random.rand(1, 1).astype(np.float32)), name="linear") ... ... def construct(self, grad, indices): ... out = self.sparse_apply_ftrl(self.var, self.accum, self.linear, grad, indices) ... return out ... >>> np.random.seed(0) >>> net = SparseApplyFtrlNet() >>> grad = Tensor(np.random.rand(1, 1).astype(np.float32)) >>> indices = Tensor(np.ones([1]), mindspore.int32) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1, 1], dtype=Float32, value= [[5.48813522e-01]]), Tensor(shape=[1, 1], dtype=Float32, value= [[7.15189338e-01]]), Tensor(shape=[1, 1], dtype=Float32, value= [[6.02763355e-01]]))
-
class
tinyms.primitives.SparseApplyFtrlV2(*args, **kwargs)[source]¶ Updates relevant entries according to the FTRL-proximal scheme. This class has one more attribute, named l2_shrinkage, than class SparseApplyFtrl.
All of inputs except indices comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
lr (float) – The learning rate value, must be positive.
l1 (float) – l1 regularization strength, must be greater than or equal to zero.
l2 (float) – l2 regularization strength, must be greater than or equal to zero.
l2_shrinkage (float) – L2 shrinkage regularization.
lr_power (float) – Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero.
use_locking (bool) – If True, the var and accumulation tensors will be protected from being updated. Default: False.
- Inputs:
var (Parameter) - The variable to be updated. The data type must be float16 or float32.
accum (Parameter) - The accumulation to be updated, must be same data type and shape as var.
linear (Parameter) - the linear coefficient to be updated, must be same data type and shape as var.
grad (Tensor) - A tensor of the same type as var, for the gradient.
indices (Tensor) - A vector of indices in the first dimension of var and accum. The shape of indices must be the same as grad in the first dimension. The type must be int32.
- Outputs:
Tuple of 3 Tensor, the updated parameters.
var (Tensor) - Tensor, has the same shape and data type as var.
accum (Tensor) - Tensor, has the same shape and data type as accum.
linear (Tensor) - Tensor, has the same shape and data type as linear.
- Raises
- Supported Platforms:
Ascend
Examples
>>> import mindspore >>> import mindspore.nn as nn >>> import numpy as np >>> from mindspore import Parameter >>> from mindspore import Tensor >>> from mindspore.ops import operations as ops >>> class SparseApplyFtrlV2Net(nn.Cell): ... def __init__(self): ... super(SparseApplyFtrlV2Net, self).__init__() ... self.sparse_apply_ftrl_v2 = ops.SparseApplyFtrlV2(lr=0.01, l1=0.0, l2=0.0, ... l2_shrinkage=0.0, lr_power=-0.5) ... self.var = Parameter(Tensor(np.random.rand(1, 2).astype(np.float32)), name="var") ... self.accum = Parameter(Tensor(np.random.rand(1, 2).astype(np.float32)), name="accum") ... self.linear = Parameter(Tensor(np.random.rand(1, 2).astype(np.float32)), name="linear") ... ... def construct(self, grad, indices): ... out = self.sparse_apply_ftrl_v2(self.var, self.accum, self.linear, grad, indices) ... return out ... >>> np.random.seed(0) >>> net = SparseApplyFtrlV2Net() >>> grad = Tensor(np.random.rand(1, 2).astype(np.float32)) >>> indices = Tensor(np.ones([1]), mindspore.int32) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[1, 2], dtype=Float32, value= [[ 5.48813522e-01, 7.15189338e-01]]), Tensor(shape=[1, 2], dtype=Float32, value= [[ 6.02763355e-01, 5.44883192e-01]]), Tensor(shape=[1, 2], dtype=Float32, value= [[ 4.23654795e-01, 6.45894110e-01]]))
-
class
tinyms.primitives.SparseApplyProximalAdagrad(*args, **kwargs)[source]¶ Updates relevant entries according to the proximal adagrad algorithm. Compared with ApplyProximalAdagrad, an additional index tensor is input.
\[accum += grad * grad\]\[\text{prox_v} = var - lr * grad * \frac{1}{\sqrt{accum}}\]\[var = \frac{sign(\text{prox_v})}{1 + lr * l2} * \max(\left| \text{prox_v} \right| - lr * l1, 0)\]Inputs of var, accum and grad comply with the implicit type conversion rules to make the data types consistent. If they have different data types, lower priority data type will be converted to relatively highest priority data type. RuntimeError exception will be thrown when the data type conversion of Parameter is required.
- Parameters
use_locking (bool) – If true, the var and accum tensors will be protected from being updated. Default: False.
- Inputs:
var (Parameter) - Variable tensor to be updated. The data type must be float16 or float32.
accum (Parameter) - Variable tensor to be updated, has the same dtype as var.
lr (Union[Number, Tensor]) - The learning rate value, must be a float number or a scalar tensor with float16 or float32 data type.
l1 (Union[Number, Tensor]) - l1 regularization strength, must be a float number or a scalar tensor with float16 or float32 data type.
l2 (Union[Number, Tensor]) - l2 regularization strength, must be a float number or a scalar tensor with float16 or float32 data type..
grad (Tensor) - A tensor of the same type as var, for the gradient.
indices (Tensor) - A tensor of indices in the first dimension of var and accum. If there are duplicates in indices, the behavior is undefined. Must be one of the following types: int32, int64.
- Outputs:
Tuple of 2 tensors, the updated parameters.
var (Tensor) - The same shape and data type as var.
accum (Tensor) - The same shape and data type as accum.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> import numpy as np >>> import mindspore.nn as nn >>> from mindspore import Tensor, Parameter >>> from mindspore.ops import operations as ops >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.sparse_apply_proximal_adagrad = ops.SparseApplyProximalAdagrad() ... self.var = Parameter(Tensor(np.array([[4.1, 7.2], [1.1, 3.0]], np.float32)), name="var") ... self.accum = Parameter(Tensor(np.array([[0, 0], [0, 0]], np.float32)), name="accum") ... self.lr = 1.0 ... self.l1 = 1.0 ... self.l2 = 0.0 ... def construct(self, grad, indices): ... out = self.sparse_apply_proximal_adagrad(self.var, self.accum, self.lr, self.l1, ... self.l2, grad, indices) ... return out ... >>> net = Net() >>> grad = Tensor(np.array([[1, 1], [1, 1]], np.float32)) >>> indices = Tensor(np.array([0, 1], np.int32)) >>> output = net(grad, indices) >>> print(output) (Tensor(shape=[2, 2], dtype=Float32, value= [[ 2.09999990e+00, 5.19999981e+00], [ 0.00000000e+00, 1.00000000e+00]]), Tensor(shape=[2, 2], dtype=Float32, value= [[ 1.00000000e+00, 1.00000000e+00], [ 1.00000000e+00, 1.00000000e+00]]))
-
class
tinyms.primitives.SparseGatherV2(*args, **kwargs)[source]¶ Returns a slice of input tensor based on the specified indices and axis.
- Inputs:
input_params (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). The original Tensor.
input_indices (Tensor) - The shape of tensor is \((y_1, y_2, ..., y_S)\). Specifies the indices of elements of the original Tensor, must be in the range [0, input_param.shape[axis]).
axis (int) - Specifies the dimension index to gather indices.
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Raises
TypeError – If axis is not an int.
- Supported Platforms:
AscendGPU
Examples
>>> input_params = Tensor(np.array([[1, 2, 7, 42], [3, 4, 54, 22], [2, 2, 55, 3]]), mindspore.float32) >>> input_indices = Tensor(np.array([1, 2]), mindspore.int32) >>> axis = 1 >>> out = ops.SparseGatherV2()(input_params, input_indices, axis) >>> print(out) [[2. 7.] [4. 54.] [2. 55.]]
-
class
tinyms.primitives.SparseSoftmaxCrossEntropyWithLogits(*args, **kwargs)[source]¶ Computes the softmax cross-entropy value between logits and sparse encoding labels.
Note
Sets input logits as X, input label as Y, output as loss. Then,
\[p_{ij} = softmax(X_{ij}) = \frac{\exp(x_i)}{\sum_{j = 0}^{N-1}\exp(x_j)}\]\[loss_{ij} = \begin{cases} -ln(p_{ij}), &j = y_i \cr -ln(1 - p_{ij}), & j \neq y_i \end{cases}\]\[loss = \sum_{ij} loss_{ij}\]- Parameters
is_grad (bool) – If true, this operation returns the computed gradient. Default: False.
- Inputs:
logits (Tensor) - Input logits, with shape \((N, C)\). Data type must be float16 or float32.
labels (Tensor) - Ground truth labels, with shape \((N)\). Data type must be int32 or int64.
- Outputs:
Tensor, if is_grad is False, the output tensor is the value of loss which is a scalar tensor; if is_grad is True, the output tensor is the gradient of input with the same shape as logits.
- Raises
TypeError – If is_grad is not a bool.
TypeError – If dtype of logits is neither float16 nor float32.
TypeError – If dtype of labels is neither int32 nor int64.
ValueError – If logits_shape[0] != labels_shape[0].
- Supported Platforms:
GPUCPU
Examples
>>> logits = Tensor([[2, 3, 1, 4, 5], [2, 1, 2, 4, 3]], mindspore.float32) >>> labels = Tensor([0, 1], mindspore.int32) >>> sparse_softmax_cross = ops.SparseSoftmaxCrossEntropyWithLogits() >>> loss = sparse_softmax_cross(logits, labels) >>> print(loss) 3.4878292 >>> sparse_softmax_cross_grad = ops.SparseSoftmaxCrossEntropyWithLogits(is_grad=True) >>> loss_grad = sparse_softmax_cross_grad(logits, labels) >>> print(loss_grad) [[-0.48415753 0.04306427 0.00582811 0.11706084 0.3182043 ] [ 0.04007946 -0.4852556 0.04007946 0.2961494 0.10894729]]
-
class
tinyms.primitives.SparseToDense(*args, **kwargs)[source]¶ Converts a sparse representation into a dense tensor.
- Inputs:
indices (Tensor) - The indices of sparse representation.
values (Tensor) - Values corresponding to each row of indices.
dense_shape (tuple) - An int tuple which specifies the shape of dense tensor.
- Returns
Tensor, the shape of tensor is dense_shape.
Examples
>>> indices = Tensor([[0, 1], [1, 2]]) >>> values = Tensor([1, 2], dtype=ms.float32) >>> dense_shape = (3, 4) >>> out = ops.SparseToDense()(indices, values, dense_shape)
-
class
tinyms.primitives.Split(*args, **kwargs)[source]¶ Splits the input tensor into output_num of tensors along the given axis and output numbers.
The input_x tensor will be split into equally sized sub-tensors. This requires that input_x.shape(axis) is divisible by output_num.
- Parameters
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
tuple[Tensor], the shape of each output tensor is the same, which is \((y_1, y_2, ..., y_S)\).
- Raises
TypeError – If axis or output_num is not an int.
ValueError – If axis is out of the range [-len(input_x.shape), len(input_x.shape)), or if the output_num is less than or equal to 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> split = ops.Split(1, 2) >>> x = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]), mindspore.int32) >>> output = split(x) >>> print(output) (Tensor(shape=[2, 2], dtype=Int32, value= [[1, 1], [2, 2]]), Tensor(shape=[2, 2], dtype=Int32, value= [[1, 1], [2, 2]]))
-
class
tinyms.primitives.Sqrt(*args, **kwargs)[source]¶ Returns square root of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor whose dtype is number.
- Outputs:
Tensor, has the same shape as the input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 4.0, 9.0]), mindspore.float32) >>> sqrt = ops.Sqrt() >>> output = sqrt(input_x) >>> print(output) [1. 2. 3.]
-
class
tinyms.primitives.Square(*args, **kwargs)[source]¶ Returns square of a tensor element-wise.
- Inputs:
input_x (Tensor) - The input tensor whose dtype is number.
- Outputs:
Tensor, has the same shape and dtype as the input_x.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> square = ops.Square() >>> output = square(input_x) >>> print(output) [1. 4. 9.]
-
class
tinyms.primitives.SquareSumAll(*args, **kwargs)[source]¶ Returns the square sum of a tensor element-wise
- Inputs:
input_x1 (Tensor) - The input tensor. The data type must be float16 or float32.
input_x2 (Tensor) - The input tensor has the same type and shape as the input_x1.
Note
SquareSumAll only supports float16 and float32 data type.
- Outputs:
output_y1 (Tensor) - The same type as the input_x1.
output_y2 (Tensor) - The same type as the input_x1.
- Raises
TypeError – If neither input_x1 nor input_x2 is a Tensor.
ValueError – If input_x1 and input_x2 are not the same shape.
- Supported Platforms:
AscendGPU
Examples
>>> input_x1 = Tensor(np.array([0, 0, 2, 0]), mindspore.float32) >>> input_x2 = Tensor(np.array([0, 0, 2, 4]), mindspore.float32) >>> square_sum_all = ops.SquareSumAll() >>> output = square_sum_all(input_x1, input_x2) >>> print(output) (Tensor(shape=[], dtype=Float32, value= 4), Tensor(shape=[], dtype=Float32, value= 20))
-
class
tinyms.primitives.SquaredDifference(*args, **kwargs)[source]¶ Subtracts the second input tensor from the first input tensor element-wise and returns square of it.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is float16, float32, int32 or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor or a tensor whose data type is float16, float32, int32 or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – if input_x and input_y is not a Number or a bool or a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> input_y = Tensor(np.array([2.0, 4.0, 6.0]), mindspore.float32) >>> squared_difference = ops.SquaredDifference() >>> output = squared_difference(input_x, input_y) >>> print(output) [1. 4. 9.]
-
class
tinyms.primitives.Squeeze(*args, **kwargs)[source]¶ Returns a tensor with the same type but dimensions of 1 are removed based on axis.
If axis is specified, it will remove the dimensions of size 1 in the given axis. It axis is None, it will remove all the dimensions of size 1.
Note
The dimension index starts at 0 and must be in the range [-input.ndim, input.ndim].
- Parameters
axis (Union[int, tuple(int)]) – Specifies the dimension indexes of shape to be removed, which will remove all the dimensions that are equal to 1. If specified, it must be int32 or int64. Default: (), an empty tuple.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
- Outputs:
Tensor, the shape of tensor is \((x_1, x_2, ..., x_S)\).
- Raises
TypeError – If axis is neither an int nor tuple.
TypeError – If axis is a tuple whose elements are not all int.
ValueError – If the corresponding dimension of the specified axis does not equal to 1.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.ones(shape=[3, 2, 1]), mindspore.float32) >>> squeeze = ops.Squeeze(2) >>> output = squeeze(input_tensor) >>> print(output) [[1. 1.] [1. 1.] [1. 1.]]
-
class
tinyms.primitives.Stack(*args, **kwargs)[source]¶ Stacks a list of tensors in specified axis.
Stacks the list of input tensors with the same rank R, output is a tensor of rank (R+1).
Given input tensors of shape \((x_1, x_2, ..., x_R)\). Set the number of input tensors as N. If \(0 \le axis\), the shape of the output tensor is \((x_1, x_2, ..., x_{axis}, N, x_{axis+1}, ..., x_R)\).
- Parameters
axis (int) – Dimension to stack. Default: 0. Negative values wrap around. The range is [-(R+1), R+1).
- Inputs:
input_x (Union[tuple, list]) - A Tuple or list of Tensor objects with the same shape and type.
- Outputs:
Tensor. A stacked Tensor with the same type as input_x.
- Raises
TypeError – If the data types of elements in input_x are not the same.
ValueError – If the length of input_x is not greater than 1; or if axis is out of the range [-(R+1), R+1); or if the shapes of elements in input_x are not the same.
- Supported Platforms:
AscendGPUCPU
Examples
>>> data1 = Tensor(np.array([0, 1]).astype(np.float32)) >>> data2 = Tensor(np.array([2, 3]).astype(np.float32)) >>> stack = ops.Stack() >>> output = stack([data1, data2]) >>> print(output) [[0. 1.] [2. 3.]]
-
class
tinyms.primitives.StandardLaplace(*args, **kwargs)[source]¶ Generates random numbers according to the Laplace random number distribution (mean=0, lambda=1). It is defined as:
\[\text{f}(x;0,1) = \frac{1}{2}\exp(-|x|),\]- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
- Outputs:
Tensor. The shape that the input ‘shape’ denotes. The dtype is float32.
- Raises
TypeError – If neither seed nor seed2 is an int.
TypeError – If shape is not a tuple.
ValueError – If shape is not a constant value.
- Supported Platforms:
Ascend
Examples
>>> shape = (4, 16) >>> stdlaplace = ops.StandardLaplace(seed=2) >>> output = stdlaplace(shape) >>> result = output.shape >>> print(result) (4, 16)
-
class
tinyms.primitives.StandardNormal(*args, **kwargs)[source]¶ Generates random numbers according to the standard Normal (or Gaussian) random number distribution.
- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
- Outputs:
Tensor. The shape is the same as the input shape. The dtype is float32.
- Raises
TypeError – If neither seed nor seed2 is an int.
TypeError – If shape is not a tuple.
ValueError – If shape is not a constant value.
- Supported Platforms:
AscendGPUCPU
Examples
>>> shape = (4, 16) >>> stdnormal = ops.StandardNormal(seed=2) >>> output = stdnormal(shape) >>> result = output.shape >>> print(result) (4, 16)
-
class
tinyms.primitives.StridedSlice(*args, **kwargs)[source]¶ Extracts a strided slice of a tensor.
Given an input tensor, this operation inserts a dimension of length 1 at the dimension. This operation extracts a fragment of size (end-begin)/stride from the given ‘input_tensor’. Starting from the beginning position, the fragment continues adding stride to the index until all dimensions are not less than the ending position.
Given a input_x[m1, m2, …, mn], begin, end and strides will be vectors of length n.
In each mask field (begin_mask, end_mask, ellipsis_mask, new_axis_mask, shrink_axis_mask) the ith bit will correspond to the ith m.
If the ith bit of begin_mask is set, begin[i] is ignored and the fullest possible range in that dimension is used instead. end_mask is analogous, except with the end range.
As for a 5*6*7 tensor, x[2:,:3,:] is equivalent to x[2:5,0:3,0:7].
If the ith bit of ellipsis_mask is set, as many unspecified dimensions as needed will be inserted between other dimensions. Only one non-zero bit is allowed in ellipsis_mask.
As for a 5*6*7*8 tensor, x[2:,…,:6] is equivalent to x[2:5,:,:,0:6]. x[2:,…] is equivalent to x[2:5,:,:,:].
If the ith bit of new_axis_mask is set, begin, end and strides are ignored and a new length 1 dimension is added at the specified position in tthe output tensor.
As for a 5*6*7 tensor, x[:2, newaxis, :6] will produce a tensor with shape (2, 1, 7).
If the ith bit of shrink_axis_mask is set, ith size shrinks the dimension by 1, taking on the value at index begin[i], end[i] and strides[i] are ignored.
As for a 5*6*7 tensor, x[:, 5, :] will result in shrink_axis_mask equal to 4.
Note
The stride may be negative value, which causes reverse slicing. The shape of begin, end and strides must be the same. begin and end are zero-indexed. The element of strides must be non-zero.
- Parameters
- Inputs:
input_x (Tensor) - The input Tensor.
begin (tuple[int]) - A tuple which represents the location where to start. Only constant value is allowed.
end (tuple[int]) - A tuple or which represents the maximum location where to end. Only constant value is allowed.
strides (tuple[int]) - A tuple which represents the stride is continuously added before reaching the maximum location. Only constant value is allowed.
- Outputs:
Tensor, The output is explained by following example.
In the 0th dimension, begin is 1, end is 2, and strides is 1, because \(1+1=2\geq2\), the interval is \([1,2)\). Thus, return the element with \(index = 1\) in 0th dimension, i.e., [[3, 3, 3], [4, 4, 4]].
In the 1st dimension, similarly, the interval is \([0,1)\). Based on the return value of the 0th dimension, return the element with \(index = 0\), i.e., [3, 3, 3].
In the 2nd dimension, similarly, the interval is \([0,3)\). Based on the return value of the 1st dimension, return the element with \(index = 0,1,2\), i.e., [3, 3, 3].
Finally, the output is [3, 3, 3].
- Raises
TypeError – If begin_mask, end_mask, ellipsis_mask, new_axis_mask or shrink_axis_mask is not an int.
TypeError – If begin, end or strides is not a tuple.
ValueError – If begin_mask, end_mask, ellipsis_mask, new_axis_mask or shrink_axis_mask is less than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor([[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]], ... [[5, 5, 5], [6, 6, 6]]], mindspore.float32) >>> slice = ops.StridedSlice() >>> output = slice(input_x, (1, 0, 0), (2, 1, 3), (1, 1, 1)) >>> print(output) [[[3. 3. 3.]]]
-
class
tinyms.primitives.Sub(*args, **kwargs)[source]¶ Subtracts the second input tensor from the first input tensor element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not a Number or a bool or a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3]), mindspore.int32) >>> input_y = Tensor(np.array([4, 5, 6]), mindspore.int32) >>> sub = ops.Sub() >>> output = sub(input_x, input_y) >>> print(output) [-3 -3 -3]
-
class
tinyms.primitives.Tan(*args, **kwargs)[source]¶ Computes tangent of input_x element-wise.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\). Data type must be float16, float32 or int32.
- Outputs:
Tensor, has the same shape as input_x.
- Raises
- Supported Platforms:
AscendCPU
Examples
>>> tan = ops.Tan() >>> input_x = Tensor(np.array([-1.0, 0.0, 1.0]), mindspore.float32) >>> output = tan(input_x) >>> print(output) [-1.5574081 0. 1.5574081]
-
class
tinyms.primitives.Tanh(*args, **kwargs)[source]¶ Tanh activation function.
Computes hyperbolic tangent of input element-wise. The Tanh function is defined as:
\[tanh(x_i) = \frac{\exp(x_i) - \exp(-x_i)}{\exp(x_i) + \exp(-x_i)} = \frac{\exp(2x_i) - 1}{\exp(2x_i) + 1},\]where \(x_i\) is an element of the input Tensor.
- Inputs:
input_x (Tensor) - The input of Tanh with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_x.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 4, 5]), mindspore.float32) >>> tanh = ops.Tanh() >>> output = tanh(input_x) >>> print(output) [0.7615941 0.9640276 0.9950547 0.9993293 0.9999092]
-
class
tinyms.primitives.TensorAdd(**kwargs)[source]¶ Same as operator Add. TensorAdd will be deprecated in the future. Please use Add instead.
-
class
tinyms.primitives.TensorScatterUpdate(*args, **kwargs)[source]¶ Creates a new tensor by updating the positions in input_x indicicated by indices, with values from update. This operation is almost equivalent to using ScatterNd, except that the updates are applied on input_x instead of a zero tensor.
indices must have rank at least 2, the last axis is the depth of each index vectors. For each index vector, there must be a corresponding value in update. If the depth of each index tensor matches the rank of input_x, then each index vector corresponds to a scalar in input_x and each update updates a scalar. If the depth of each index tensor is less than the rank of input_x, then each index vector corresponds to a slice in input_x, and each update updates a slice.
The order in which updates are applied is nondeterministic, meaning that if there are multiple index vectors in indices that correspond to the same position, the value of that position in the output will be nondeterministic.
- Inputs:
input_x (Tensor) - The target tensor. The dimension of input_x must be no less than indices.shape[-1].
indices (Tensor) - The index of input tensor whose data type is int32 or int64. The rank must be at least 2.
update (Tensor) - The tensor to update the input tensor, has the same type as input, and update.shape = indices.shape[:-1] + input_x.shape[indices.shape[-1]:].
- Outputs:
Tensor, has the same shape and type as input_x.
- Raises
TypeError – If dtype of indices is neither int32 nor int64.
ValueError – If length of shape of input_x is less than the last dimension of shape of indices.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6], [0.4, 0.5, -3.2]]), mindspore.float32) >>> indices = Tensor(np.array([[0, 0], [1, 1]]), mindspore.int32) >>> update = Tensor(np.array([1.0, 2.2]), mindspore.float32) >>> op = ops.TensorScatterUpdate() >>> output = op(input_x, indices, update) >>> print(output) [[ 1. 0.3 3.6] [ 0.4 2.2 -3.2]]
-
class
tinyms.primitives.TensorSummary(*args, **kwargs)[source]¶ Outputs a tensor to a protocol buffer through a tensor summary operator.
- Inputs:
name (str) - The name of the input variable.
value (Tensor) - The value of tensor, and the rank of tensor must be greater than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> import mindspore.nn as nn >>> import mindspore.ops as ops >>> >>> >>> class SummaryDemo(nn.Cell): ... def __init__(self,): ... super(SummaryDemo, self).__init__() ... self.summary = ops.TensorSummary() ... self.add = ops.Add() ... ... def construct(self, x, y): ... x = self.add(x, y) ... name = "x" ... self.summary(name, x) ... return x ...
-
class
tinyms.primitives.Tile(*args, **kwargs)[source]¶ Replicates a tensor with given multiples times.
Creates a new tensor by replicating input_x multiples times. The i’th dimension of output tensor has input_x.shape(i) * multiples[i] elements, and the values of input_x are replicated multiples[i] times along the i’th dimension.
- Inputs:
input_x (Tensor) - 1-D or higher Tensor. Set the shape of input tensor as \((x_1, x_2, ..., x_S)\).
multiples (tuple[int]) - The input tuple is constructed by multiple integers, i.e., \((y_1, y_2, ..., y_S)\). The length of multiples cannot be smaller than the length of the shape of input_x. Only constant value is allowed.
- Outputs:
Tensor, has the same data type as the input_x.
If the length of multiples is the same as the length of shape of input_x, then the shape of their corresponding positions can be multiplied, and the shape of Outputs is \((x_1*y_1, x_2*y_2, ..., x_S*y_R)\).
If the length of multiples is larger than the length of shape of input_x, fill in multiple 1 in the length of the shape of input_x until their lengths are consistent. Such as set the shape of input_x as \((1, ..., x_1, x_2, ..., x_S)\), then the shape of their corresponding positions can be multiplied, and the shape of Outputs is \((1*y_1, ..., x_S*y_R)\).
- Raises
TypeError – If multiples is not a tuple or its elements are not all int.
ValueError – If the elements of multiples are not all greater than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> tile = ops.Tile() >>> input_x = Tensor(np.array([[1, 2], [3, 4]]), mindspore.float32) >>> multiples = (2, 3) >>> output = tile(input_x, multiples) >>> print(output) [[1. 2. 1. 2. 1. 2.] [3. 4. 3. 4. 3. 4.] [1. 2. 1. 2. 1. 2.] [3. 4. 3. 4. 3. 4.]]
-
class
tinyms.primitives.TopK(*args, **kwargs)[source]¶ Finds values and indices of the k largest entries along the last dimension.
- Parameters
sorted (bool) – If true, the obtained elements will be sorted by the values in descending order. Default: False.
- Inputs:
input_x (Tensor) - Input to be computed, data type must be float16, float32 or int32.
k (int) - The number of top elements to be computed along the last dimension, constant input is needed.
- Outputs:
Tuple of 2 tensors, the values and the indices.
values (Tensor) - The k largest elements in each slice of the last dimensional.
indices (Tensor) - The indices of values within the last dimension of input.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> topk = ops.TopK(sorted=True) >>> input_x = Tensor([1, 2, 3, 4, 5], mindspore.float16) >>> k = 3 >>> values, indices = topk(input_x, k) >>> print((values, indices)) (Tensor(shape=[3], dtype=Float16, value= [ 5.0000e+00, 4.0000e+00, 3.0000e+00]), Tensor(shape=[3], dtype=Int32, value= [4, 3, 2]))
-
class
tinyms.primitives.TransferCrd(*args, **kwargs)[source]¶ Transfer the coordinates to angular and radial.
- Parameters
start_serial (int32) – the index start position.
end_serial (int32) – the index end position.
number (int32) – the length of angular and radial.
- Inputs:
crd (Tensor, float32) - [N, 3], the coordinate of each atom. N is the number of atoms.
- Outputs:
output (uint32)
- Supported Platforms:
GPU
-
class
tinyms.primitives.Transpose(*args, **kwargs)[source]¶ Permutes the dimensions of the input tensor according to input permutation.
- Inputs:
input_x (Tensor) - The shape of tensor is \((x_1, x_2, ..., x_R)\).
input_perm (tuple[int]) - The permutation to be converted. The input tuple is constructed by multiple indexes. The length of input_perm and the shape of input_x must be the same. Only constant value is allowed. Must be in the range [0, rank(input_x)).
- Outputs:
Tensor, the type of output tensor is the same as input_x and the shape of output tensor is decided by the shape of input_x and the value of input_perm.
- Raises
TypeError – If input_perm is not a tuple.
ValueError – If length of shape of input_x is not equal to length of shape of input_perm.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_tensor = Tensor(np.array([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]]), mindspore.float32) >>> perm = (0, 2, 1) >>> transpose = ops.Transpose() >>> output = transpose(input_tensor, perm) >>> print(output) [[[ 1. 4.] [ 2. 5.] [ 3. 6.]] [[ 7. 10.] [ 8. 11.] [ 9. 12.]]]
-
class
tinyms.primitives.TruncateDiv(*args, **kwargs)[source]¶ Divides the first input tensor by the second input tensor element-wise for integer types, negative numbers will round fractional quantities towards zero.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.int32) >>> input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) >>> truncate_div = ops.TruncateDiv() >>> output = truncate_div(input_x, input_y) >>> print(output) [0 1 0]
-
class
tinyms.primitives.TruncateMod(*args, **kwargs)[source]¶ Returns the remainder of division element-wise.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is number or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is number or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If neither input_x nor input_y is one of the following: Tensor, Number, bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.int32) >>> input_y = Tensor(np.array([3, 3, 3]), mindspore.int32) >>> truncate_mod = ops.TruncateMod() >>> output = truncate_mod(input_x, input_y) >>> print(output) [ 2 1 -1]
-
class
tinyms.primitives.TruncatedNormal(*args, **kwargs)[source]¶ Returns a tensor of the specified shape filled with truncated normal values.
The generated values follow a normal distribution.
- Parameters
seed (int) – A integer number used to create random seed. Default: 0.
dtype (
mindspore.dtype) – Data type. Default: mindspore.float32.
- Inputs:
shape (tuple[int]) - The shape of the output tensor, is a tuple of positive integer.
- Outputs:
Tensor, the data type of output tensor is the same as attribute dtype.
Examples
>>> shape = (1, 2, 3) >>> truncated_normal = ops.TruncatedNormal() >>> output = truncated_normal(shape)
-
class
tinyms.primitives.TupleToArray(*args, **kwargs)[source]¶ Converts a tuple to a tensor.
If the type of the first number in the tuple is integer, the data type of the output tensor is int. Otherwise, the data type of the output tensor is float.
- Inputs:
input_x (tuple) - A tuple of numbers. These numbers have the same type. Only constant value is allowed.
- Outputs:
Tensor, if the input tuple contains N numbers, then the shape of the output tensor is (N,).
- Raises
TypeError – If input_x is not a tuple.
ValueError – If length of input_x is less than or equal to 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> type = ops.TupleToArray()((1,2,3)) >>> print(type) [1 2 3]
-
class
tinyms.primitives.UniformCandidateSampler(*args, **kwargs)[source]¶ Uniform candidate sampler.
This function samples a set of classes(sampled_candidates) from [0, range_max-1] based on uniform distribution. If unique=True, candidates are drawn without replacement, else unique=False with replacement.
- Parameters
num_true (int) – The number of target classes in each training example.
num_sampled (int) – The number of classes to randomly sample. The sampled_candidates will have a shape of num_sampled. If unique=True, num_sampled must be less than or equal to range_max.
unique (bool) – Whether all sampled classes in a batch are unique.
range_max (int) – The number of possible classes, must be non-negative.
seed (int) – Used for random number generation, must be non-negative. If seed has a value of 0, seed will be replaced with a randomly generated value. Default: 0.
remove_accidental_hits (bool) – Whether accidental hit is removed. Default: False.
- Inputs:
true_classes (Tensor) - A Tensor. The target classes with a Tensor shape of (batch_size, num_true).
- Outputs:
sampled_candidates (Tensor) - The sampled_candidates is independent of the true classes. Shape: (num_sampled, ).
true_expected_count (Tensor) - The expected counts under the sampling distribution of each of true_classes. Shape: (batch_size, num_true).
sampled_expected_count (Tensor) - The expected counts under the sampling distribution of each of sampled_candidates. Shape: (num_sampled, ).
- Raises
- Supported Platforms:
GPU
Examples
>>> sampler = ops.UniformCandidateSampler(1, 3, False, 4) >>> output1, output2, output3 = sampler(Tensor(np.array([[1], [3], [4], [6], [3]], dtype=np.int32))) >>> print(output1, output2, output3) [1, 1, 3], [[0.75], [0.75], [0.75], [0.75], [0.75]], [0.75, 0.75, 0.75]
-
class
tinyms.primitives.UniformInt(*args, **kwargs)[source]¶ Produces random integer values i, uniformly distributed on the closed interval [minval, maxval), that is, distributed according to the discrete probability function:
\[\text{P}(i|a,b) = \frac{1}{b-a+1},\]Note
The number in tensor minval must be strictly less than maxval at any position after broadcasting.
- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
minval (Tensor) - The distribution parameter, a. It defines the minimum possibly generated value, with int32 data type. Only one number is supported.
maxval (Tensor) - The distribution parameter, b. It defines the maximum possibly generated value, with int32 data type. Only one number is supported.
- Raises
TypeError – If neither seed nor seed2 is an int.
TypeError – If shape is not a tuple.
TypeError – If neither minval nor maxval is a Tensor.
ValueError – If shape is not a constant value.
- Outputs:
Tensor. The shape is the same as the input ‘shape’, and the data type is int32.
- Supported Platforms:
AscendGPU
Examples
>>> shape = (2, 4) >>> minval = Tensor(1, mstype.int32) >>> maxval = Tensor(5, mstype.int32) >>> uniform_int = ops.UniformInt(seed=10) >>> output = uniform_int(shape, minval, maxval) >>> result = output.shape >>> print(result) (2, 4)
-
class
tinyms.primitives.UniformReal(*args, **kwargs)[source]¶ Produces random floating-point values i, uniformly distributed to the interval [0, 1).
- Parameters
- Inputs:
shape (tuple) - The shape of random tensor to be generated. Only constant value is allowed.
- Outputs:
Tensor. The shape that the input ‘shape’ denotes. The dtype is float32.
- Raises
TypeError – If neither seed nor seed2 is an int.
TypeError – If shape is not a tuple.
ValueError – If shape is not a constant value.
- Supported Platforms:
AscendGPU
Examples
>>> shape = (2, 2) >>> uniformreal = ops.UniformReal(seed=2) >>> output = uniformreal(shape) >>> result = output.shape >>> print(result) (2, 2)
-
class
tinyms.primitives.Unique(*args, **kwargs)[source]¶ Returns the unique elements of input tensor and also return a tensor containing the index of each value of input tensor corresponding to the output unique tensor.
- Inputs:
x (Tensor) - The input tensor.
- Outputs:
Tuple, containing Tensor objects (y, idx), `y is a tensor with the same type as x, and contains the unique elements in x, sorted in ascending order. idx is a tensor containing indices of elements in the input corresponding to the output tensor.
- Raises
TypeError – If x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([1, 2, 5, 2]), mindspore.int32) >>> output = ops.Unique()(x) >>> print(output) (Tensor(shape=[3], dtype=Int32, value= [1, 2, 5]), Tensor(shape=[4], dtype=Int32, value= [0, 1, 2, 1])) >>> >>> # note that for GPU, this operator must be wrapped inside a model, and executed in graph mode. >>> class UniqueNet(nn.Cell): ... def __init__(self): ... super(UniqueNet, self).__init__() ... self.unique_op = ops.Unique() ... ... def construct(self, x): ... output, indices = self.unique_op(x) ... return output, indices ... >>> x = Tensor(np.array([1, 2, 5, 2]), mindspore.int32) >>> net = UniqueNet() >>> output = net(x) >>> print(output) (Tensor(shape=[3], dtype=Int32, value= [1, 2, 5]), Tensor(shape=[4], dtype=Int32, value= [0, 1, 2, 1]))
-
class
tinyms.primitives.UniqueWithPad(*args, **kwargs)[source]¶ Returns unique elements and relative indexes in 1-D tensor, filled with padding num.
- Inputs:
x (Tensor) - The tensor need to be unique. Must be 1-D vector with types: int32, int64.
pad_num (int) - Pad num.
- Outputs:
tuple(Tensor), tuple of 2 tensors, y and idx. - y (Tensor) - The unique elements filled with pad_num, the shape and type same as x. - idx (Tensor) - The index of each value of x in the unique output y, the shape and type same as x.
- Raises
TypeError – If dtype of x is neither int32 nor int64.
ValueError – If length of shape of x is not equal to 1.
- Supported Platforms:
AscendCPU
Examples
>>> x = Tensor(np.array([1, 1, 5, 5, 4, 4, 3, 3, 2, 2,]), mindspore.int32) >>> pad_num = 8 >>> output = ops.UniqueWithPad()(x, pad_num) >>> print(output) (Tensor(shape=[10], dtype=Int32, value= [1, 5, 4, 3, 2, 8, 8, 8, 8, 8]), Tensor(shape=[10], dtype=Int32, value= [0, 0, 1, 1, 2, 2, 3, 3, 4, 4]))
-
class
tinyms.primitives.Unpack(**kwargs)[source]¶ Same as operator Unstack. Unpack will be deprecated in the future. Please use Unstack instead.
-
class
tinyms.primitives.UnsortedSegmentMax(*args, **kwargs)[source]¶ Computes the maximum along segments of a tensor.
Note
If the segment_id i is absent in the segment_ids, then output[i] will be filled with the minimum value of the input_x’s type.
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\). The data type must be float16, float32 or int32.
segment_ids (Tensor) - A 1-D tensor whose shape is \((x_1)\), the value must be >= 0. The data type must be int32.
num_segments (int) - The value specifies the number of distinct segment_ids.
- Outputs:
Tensor, set the number of num_segments as N, the shape is \((N, x_2, ..., x_R)\).
- Raises
TypeError – If num_segments is not an int.
ValueError – If length of shape of segment_ids is not equal to 1.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[1, 2, 3], [4, 5, 6], [4, 2, 1]]).astype(np.float32)) >>> segment_ids = Tensor(np.array([0, 1, 1]).astype(np.int32)) >>> num_segments = 2 >>> unsorted_segment_max = ops.UnsortedSegmentMax() >>> output = unsorted_segment_max(input_x, segment_ids, num_segments) >>> print(output) [[1. 2. 3.] [4. 5. 6.]]
-
class
tinyms.primitives.UnsortedSegmentMin(*args, **kwargs)[source]¶ Computes the minimum of a tensor along segments.
Note
If the segment_id i is absent in the segment_ids, then output[i] will be filled with the maximum value of the input_x’s type.
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\). The data type must be float16, float32 or int32.
segment_ids (Tensor) - A 1-D tensor whose shape is \((x_1)\), the value must be >= 0. The data type must be int32.
num_segments (int) - The value specifies the number of distinct segment_ids.
- Outputs:
Tensor, set the number of num_segments as N, the shape is \((N, x_2, ..., x_R)\).
- Raises
TypeError – If num_segments is not an int.
ValueError – If length of shape of segment_ids is not equal to 1.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[1, 2, 3], [4, 5, 6], [4, 2, 1]]).astype(np.float32)) >>> segment_ids = Tensor(np.array([0, 1, 1]).astype(np.int32)) >>> num_segments = 2 >>> unsorted_segment_min = ops.UnsortedSegmentMin() >>> output = unsorted_segment_min(input_x, segment_ids, num_segments) >>> print(output) [[1. 2. 3.] [4. 2. 1.]]
-
class
tinyms.primitives.UnsortedSegmentProd(*args, **kwargs)[source]¶ Computes the product of a tensor along segments.
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\). With float16, float32 or int32 data type.
segment_ids (Tensor) - A 1-D tensor whose shape is \((x_1)\), the value must be >= 0. Data type must be int32.
num_segments (int) - The value specifies the number of distinct segment_ids, must be greater than 0.
- Outputs:
Tensor, set the number of num_segments as N, the shape is \((N, x_2, ..., x_R)\).
- Raises
TypeError – If num_segments is not an int.
ValueError – If length of shape of segment_ids is not equal to 1.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[1, 2, 3], [4, 5, 6], [4, 2, 1]]).astype(np.float32)) >>> segment_ids = Tensor(np.array([0, 1, 0]).astype(np.int32)) >>> num_segments = 2 >>> unsorted_segment_prod = ops.UnsortedSegmentProd() >>> output = unsorted_segment_prod(input_x, segment_ids, num_segments) >>> print(output) [[4. 4. 3.] [4. 5. 6.]]
-
class
tinyms.primitives.UnsortedSegmentSum(*args, **kwargs)[source]¶ Computes the sum of a tensor along segments.
Calculates a tensor such that \(\text{output}[i] = \sum_{segment\_ids[j] == i} \text{data}[j, \ldots]\), where \(j\) is a tuple describing the index of element in data. segment_ids selects which elements in data to sum up. Segment_ids does not need to be sorted, and it does not need to cover all values in the entire valid value range.
Note
If the segment_id i is absent in the segment_ids, then output[i] will be filled with 0.
If the sum of the given segment_ids \(i\) is empty, then \(\text{output}[i] = 0\). If the given segment_ids is negative, the value will be ignored. ‘num_segments’ must be equal to the number of different segment_ids.
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\).
segment_ids (Tensor) - Set the shape as \((x_1, x_2, ..., x_N)\), where 0 < N <= R. Type must be int.
num_segments (int) - Set \(z\) as num_segments.
- Outputs:
Tensor, the shape is \((z, x_{N+1}, ..., x_R)\).
- Raises
TypeError – If num_segments is not an int.
ValueError – If length of shape of segment_ids is less than 1.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor([1, 2, 3, 4], mindspore.float32) >>> segment_ids = Tensor([0, 0, 1, 2], mindspore.int32) >>> num_segments = 4 >>> output = ops.UnsortedSegmentSum()(input_x, segment_ids, num_segments) >>> print(output) [3. 3. 4. 0.]
-
class
tinyms.primitives.Unstack(*args, **kwargs)[source]¶ Unstacks tensor in specified axis.
Unstacks a tensor of rank R along axis dimension, output tensors will have rank (R-1).
Given a tensor of shape \((x_1, x_2, ..., x_R)\). If \(0 \le axis\), the shape of tensor in output is \((x_1, x_2, ..., x_{axis}, x_{axis+2}, ..., x_R)\).
This is the opposite of pack.
- Parameters
axis (int) – Dimension along which to pack. Default: 0. Negative values wrap around. The range is [-R, R).
- Inputs:
input_x (Tensor) - The shape is \((x_1, x_2, ..., x_R)\). A tensor to be unstacked and the rank of the tensor must be greater than 0.
- Outputs:
A tuple of tensors, the shape of each objects is the same.
- Raises
ValueError – If axis is out of the range [-len(input_x.shape), len(input_x.shape)).
- Supported Platforms:
AscendGPUCPU
Examples
>>> unstack = ops.Unstack() >>> input_x = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]])) >>> output = unstack(input_x) >>> print(output) (Tensor(shape=[4], dtype=Int64, value= [1, 1, 1, 1]), Tensor(shape=[4], dtype=Int64, value= [2, 2, 2, 2]))
-
class
tinyms.primitives.UpdateState(*args, **kwargs)[source]¶ UpdateState is used for update side-effect state.
- Inputs:
value (State) - the state value to be updated.
expr (Expression) - the expression to evaluate before state changes.
- Outputs:
State, the updated state value.
-
class
tinyms.primitives.Xdivy(*args, **kwargs)[source]¶ Divides the first input tensor by the second input tensor element-wise. Returns zero when x is zero.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number, or a bool, or a tensor whose data type is float16, float32 or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number, or a bool when the first input is a tensor, or a tensor whose data type is float16, float32 or bool.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([2, 4, -1]), mindspore.float32) >>> input_y = Tensor(np.array([2, 2, 2]), mindspore.float32) >>> xdivy = ops.Xdivy() >>> output = xdivy(input_x, input_y) >>> print(output) [ 1. 2. -0.5]
-
class
tinyms.primitives.Xlogy(*args, **kwargs)[source]¶ Computes the first input tensor multiplied by the logarithm of second input tensor element-wise. Returns zero when x is zero.
Inputs of input_x and input_y comply with the implicit type conversion rules to make the data types consistent. The inputs must be two tensors or one tensor and one scalar. When the inputs are two tensors, dtypes of them cannot be both bool, and the shapes of them could be broadcast. When the inputs are one tensor and one scalar, the scalar could only be a constant.
- Inputs:
input_x (Union[Tensor, Number, bool]) - The first input is a number or a bool or a tensor whose data type is float16, float32 or bool.
input_y (Union[Tensor, Number, bool]) - The second input is a number or a bool when the first input is a tensor or a tensor whose data type is float16, float32 or bool. The value must be positive.
- Outputs:
Tensor, the shape is the same as the one after broadcasting, and the data type is the one with higher precision or higher digits among the two inputs.
- Raises
TypeError – If input_x and input_y is not one of the following: Tensor, Number, bool.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([-5, 0, 4]), mindspore.float32) >>> input_y = Tensor(np.array([2, 2, 2]), mindspore.float32) >>> xlogy = ops.Xlogy() >>> output = xlogy(input_x, input_y) >>> print(output) [-3.465736 0. 2.7725887]
-
class
tinyms.primitives.Zeros(*args, **kwargs)[source]¶ Creates a tensor filled with value zeros.
Creates a tensor with shape described by the first argument and fills it with value zeros in type of the second argument.
- Inputs:
shape (Union[tuple[int], int]) - The specified shape of output tensor. Only constant positive int is allowed.
type (mindspore.dtype) - The specified type of output tensor. Only constant value is allowed.
- Outputs:
Tensor, has the same type and shape as input shape value.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore.ops import operations as ops >>> zeros = ops.Zeros() >>> output = zeros((2, 2), mindspore.float32) >>> print(output) [[0. 0.] [0. 0.]]
-
class
tinyms.primitives.ZerosLike(*args, **kwargs)[source]¶ Creates a new tensor. All elements value are 0.
Returns a tensor of zeros with the same shape and data type as the input tensor.
- Inputs:
input_x (Tensor) - Input tensor.
- Outputs:
Tensor, has the same shape and data type as input_x but filled with zeros.
- Raises
TypeError – If input_x is not a Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> zeroslike = ops.ZerosLike() >>> x = Tensor(np.array([[0, 1], [2, 1]]).astype(np.float32)) >>> output = zeroslike(x) >>> print(output) [[0. 0.] [0. 0.]]
tinyms.layers¶
Layer module contains pre-defined building blocks or computing units to construct neural networks.
The high-level components (Layers) used to construct the neural network.
-
class
tinyms.layers.Layer(auto_prefix=True, flags=None)[source]¶ Base class for all neural networks.
A ‘Layer’ could be a single neural network layer, such as conv2d, relu, batch_norm, etc. or a composition of cells to constructing a network.
Note
In general, the autograd algorithm will automatically generate the implementation of the gradient function, but if back-propagation(bprop) method is implemented, the gradient function will be replaced by the bprop. The bprop implementation will receive a Tensor dout containing the gradient of the loss w.r.t. the output, and a Tensor out containing the forward result. The bprop needs to compute the gradient of the loss w.r.t. the inputs, gradient of the loss w.r.t. Parameter variables are not supported currently. The bprop method must contain the self parameter.
- Parameters
auto_prefix (bool) – Recursively generate namespaces. Default: True.
Examples
>>> from tinyms import layers, primitives as P >>> >>> class MyNet(layers.Layer): ... def __init__(self): ... super(MyNet, self).__init__() ... self.relu = P.ReLU() ... ... def construct(self, x): ... return self.relu(x)
-
property
bprop_debug¶ Get whether cell custom bprop debug is enabled.
-
cast_param(param)¶ Cast parameter according to auto mix precision level in pynative mode.
- Parameters
param (Parameter) – The parameter to cast.
-
cells()¶ Returns an iterator over immediate cells.
-
cells_and_names(cells=None, name_prefix='')¶ Returns an iterator over all cells in the network.
Includes the cell’s name and itself.
- Parameters
Examples
>>> n = Net() >>> names = [] >>> for m in n.cells_and_names(): ... if m[0]: ... names.append(m[0])
-
compile_and_run(*inputs)¶ Compiles and runs cell.
- Parameters
inputs (tuple) – Input parameters.
- Returns
Object, the result of executing.
-
construct(*inputs, **kwargs)¶ Defines the computation to be performed. This method must be overridden by all subclasses.
- Returns
Tensor, returns the computed result.
-
exec_checkpoint_graph()¶ Executes saving checkpoint graph operation.
-
extend_repr()¶ Sets the extended representation of the Cell.
To print customized extended information, re-implement this method in your own cells.
-
generate_scope()¶ Generate the scope for each cell object in the network.
-
get_func_graph_proto()¶ Return graph binary proto.
-
get_parameters(expand=True)¶ Returns an iterator over cell parameters.
Yields parameters of this cell. If expand is True, yield parameters of this cell and all subcells.
- Parameters
expand (bool) – If true, yields parameters of this cell and all subcells. Otherwise, only yield parameters that are direct members of this cell. Default: True.
Examples
>>> net = Net() >>> parameters = [] >>> for item in net.get_parameters(): ... parameters.append(item)
-
get_scope()¶ Returns the scope of a cell object in one network.
-
init_parameters_data(auto_parallel_mode=False)¶ Initialize all parameters and replace the original saved parameters in cell.
Note
trainable_params() and other similar interfaces may return different parameter instance after init_parameters_data, do not save these result.
- Parameters
auto_parallel_mode (bool) – If running in auto_parallel_mode.
- Returns
Dict[Parameter, Parameter], returns a dict of original parameter and replaced parameter.
-
insert_child_to_cell(child_name, child_cell)¶ Adds a child cell to the current cell with a given name.
-
insert_param_to_cell(param_name, param, check_name=True)¶ Adds a parameter to the current cell.
Inserts a parameter with given name to the cell. Please refer to the usage in source code of mindspore.nn.Cell.__setattr__.
- Parameters
- Raises
KeyError – If the name of parameter is null or contains dot.
AttributeError – If user did not call init() first.
TypeError – If the type of parameter is not Parameter.
-
load_parameter_slice(params)¶ Replace parameters with sliced tensors by parallel strategies.
Please refer to the usage in source code of mindspore.common._Executor.compile.
- Parameters
params (dict) – The parameters dictionary used for initializing the data graph.
-
name_cells()¶ Returns an iterator over all cells in the network.
Include name of the cell and cell itself.
-
property
param_prefix¶ Param prefix is the prefix of current cell’s direct child parameter.
-
parameters_and_names(name_prefix='', expand=True)¶ Returns an iterator over cell parameters.
Includes the parameter’s name and itself.
- Parameters
Examples
>>> n = Net() >>> names = [] >>> for m in n.parameters_and_names(): ... if m[0]: ... names.append(m[0])
-
parameters_dict(recurse=True)¶ Gets parameters dictionary.
Gets the parameters dictionary of this cell.
- Parameters
recurse (bool) – Whether contains the parameters of subcells. Default: True.
- Returns
OrderedDict, return parameters dictionary.
-
recompute(mode=True)¶ Set the cell recomputed. All the primitive in the cell will be set recomputed. If a primitive set recomputed feeds into some backward nodes for computing gradient, rather than storing the intermediate activation computed in forward pass, we will recompute it in backward pass.
Note
If the computation involves something like randomization or global variable, the equivalence is not guaranteed currently.
If the recompute api of a primitive in this cell is also called, the recompute mode of this primitive is subject to the recompute api of the primitive.
- Parameters
mode (bool) – Specifies whether the cell is recomputed. Default: True.
-
register_backward_hook(fn)¶ Set the cell backward hook function. Note that this function is only supported in Pynative Mode.
Note
fn must be defined as the following code. cell_name is the name of registered cell. grad_input is gradient passed to the cell. grad_output is the gradient computed and passed to the next cell or primitive, which may be modified and returned. hook_fn(cell_name, grad_input, grad_output) -> Tensor or None.
- Parameters
fn (function) – Specifies the hook function with grad as input.
-
remove_redundant_parameters()¶ Remove the redundant parameters
-
set_auto_parallel()¶ Set the cell to auto parallel mode.
Note
If a cell needs to use the auto parallel or semi auto parallel mode for training, evaluation or prediction, this interface needs to be called by the cell.
-
set_broadcast_flag(mode=True)¶ Set the cell to data_parallel mode.
The cell can be accessed as an attribute using the given name.
- Parameters
mode (bool) – Specifies whether the model is data_parallel. Default: True.
-
set_comm_fusion(fusion_type, recurse=True)¶ Set comm_fusion for all the parameters in the Net. Please refer to the description of mindspore.common.parameter.comm_fusion.
Note
The value of attribute will be overwritten when the function is called multiply.
-
set_grad(requires_grad=True)¶ Sets the cell flag for gradient. In pynative mode, this parameter specifies whether the network require gradients. If True, the backward network needed to compute the gradients will be generated when the forward network is executed.
- Parameters
requires_grad (bool) – Specifies if the net need to grad, if it is True, cell will construct backward network in pynative mode. Default: True.
-
set_parallel_input_with_inputs(*inputs)¶ Slice inputs tensors by parallel strategies, and set the sliced inputs to _parallel_input_run
- Parameters
inputs (tuple) – inputs of construct method.
-
set_param_ps(recurse=True, init_in_server=False)¶ Set whether the trainable parameters are updated by parameter server and whether the trainable parameters are initialized on server.
Note
It only works when a running task is in the parameter server mode.
-
set_train(mode=True)¶ Sets the cell to training mode.
The cell itself and all children cells will be set to training mode. Layers that have different constructions for training and predicting, such as BatchNorm, will distinguish between the branches by this attribute. If set to True, the training branch will be executed, otherwise another branch.
- Parameters
mode (bool) – Specifies whether the model is training. Default: True.
-
to_float(dst_type)¶ Add cast on all inputs of cell and child cells to run with certain float type.
If dst_type is mindspore.dtype.float16, all the inputs of Cell including input, Parameter, Tensor as const will be cast to float16. Please refer to the usage in source code of mindspore.train.amp.build_train_network.
Note
Multiple calls will overwrite.
- Parameters
dst_type (
mindspore.dtype) – Transfer Cell to Run with dst_type. dst_type can be mindspore.dtype.float16 or mindspore.dtype.float32.- Raises
ValueError – If dst_type is not float32 nor float16.
-
trainable_params(recurse=True)¶ Returns all trainable parameters.
Returns a list of all trainable parameters.
- Parameters
recurse (bool) – Whether contains the trainable parameters of subcells. Default: True.
- Returns
List, the list of trainable parameters.
-
untrainable_params(recurse=True)¶ Returns all untrainable parameters.
Returns a list of all untrainable parameters.
- Parameters
recurse (bool) – Whether contains the untrainable parameters of subcells. Default: True.
- Returns
List, the list of untrainable parameters.
-
update_cell_prefix()¶ Update the all child cells’ self.param_prefix.
After being invoked, it can get all the cell’s children’s name prefix by ‘_param_prefix’.
-
update_cell_type(cell_type)¶ The current cell type is updated when a quantization aware training network is encountered.
After being invoked, it can set the cell type to ‘cell_type’.
Layer module contains pre-defined building blocks or computing units to construct neural networks.
The high-level components (Layers) used to construct the neural network.
-
class
tinyms.layers.SequentialLayer(*args)[source]¶ Sequential layer container.
A list of Layers will be added to it in the order they are passed in the constructor. Alternatively, an ordered dict of cells can also be passed in.
- Parameters
args (Union[list, OrderedDict]) – List of subclass of Layer.
- Raises
TypeError – If the type of the argument is not list or OrderedDict.
- Inputs:
input (Tensor) - Tensor with shape according to the first Cell in the sequence.
- Outputs:
Tensor, the output Tensor with shape depending on the input and defined sequence of Layers.
Examples
>>> import tinyms as ts >>> from tinyms.layers import SequentialLayer, Conv2d, ReLU >>> >>> seq_layer = SequentialLayer([Conv2d(3, 2, 3, pad_mode='valid', weight_init="ones"), ReLU()]) >>> x = ts.ones([1, 3, 4, 4]) >>> print(seq_layer(x)) [[[[27. 27.] [27. 27.]] [[27. 27.] [27. 27.]]]]
-
class
tinyms.layers.LayerList(*args, **kwargs)[source]¶ Holds Layers in a list.
LayerList can be used like a regular Python list, support ‘__getitem__’, ‘__setitem__’, ‘__delitem__’, ‘__len__’, ‘__iter__’ and ‘__iadd__’, but layers it contains are properly registered, and will be visible by all Layer methods.
- Parameters
args (list, optional) – List of subclass of Layer.
Examples
>>> from tinyms.layers import LayerList, Conv2d, BatchNorm2d, ReLU >>> >>> conv = nn.Conv2d(100, 20, 3) >>> layers = LayerList([BatchNorm2d(20)]) >>> layers.insert(0, Conv2d(100, 20, 3)) >>> layers.append(ReLU()) >>> layers LayerList< (0): Conv2d<input_channels=100, ..., bias_init=None> (1): BatchNorm2d<num_features=20, ..., moving_variance=Parameter (name=variance)> (2): ReLU<> >
-
class
tinyms.layers.Softmax(axis=-1)[source]¶ Softmax activation function.
Applies the Softmax function to an n-dimensional input Tensor.
The input is a Tensor of logits transformed with exponential function and then normalized to lie in range [0, 1] and sum up to 1.
Softmax is defined as:
\[\text{softmax}(x_{i}) = \frac{\exp(x_i)}{\sum_{j=0}^{n-1}\exp(x_j)},\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Parameters
axis (Union[int, tuple[int]]) – The axis to apply Softmax operation, -1 means the last dimension. Default: -1.
- Inputs:
x (Tensor) - The input of Softmax with data type of float16 or float32.
- Outputs:
Tensor, which has the same type and shape as x with values in the range[0,1].
- Raises
TypeError – If axis is neither an int nor a tuple.
TypeError – If dtype of x is neither float16 nor float32.
ValueError – If axis is a tuple whose length is less than 1.
ValueError – If axis is a tuple whose elements are not all in range [-len(x), len(x)).
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> softmax = nn.Softmax() >>> output = softmax(input_x) >>> print(output) [0.03168 0.01166 0.0861 0.636 0.2341 ]
-
class
tinyms.layers.LogSoftmax(axis=-1)[source]¶ LogSoftmax activation function.
Applies the LogSoftmax function to n-dimensional input tensor.
The input is transformed by the Softmax function and then by the log function to lie in range[-inf,0).
Logsoftmax is defined as:
\[\text{logsoftmax}(x_i) = \log \left(\frac{\exp(x_i)}{\sum_{j=0}^{n-1} \exp(x_j)}\right),\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Parameters
axis (int) – The axis to apply LogSoftmax operation, -1 means the last dimension. Default: -1.
- Inputs:
x (Tensor) - The input of LogSoftmax, with float16 or float32 data type.
- Outputs:
Tensor, which has the same type and shape as the input as x with values in the range[-inf,0).
- Raises
TypeError – If axis is not an int.
TypeError – If dtype of x is neither float16 nor float32.
ValueError – If axis is not in range [-len(x), len(x)).
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> log_softmax = nn.LogSoftmax() >>> output = log_softmax(input_x) >>> print(output) [[-5.00672150e+00 -6.72150636e-03 -1.20067215e+01] [-7.00091219e+00 -1.40009127e+01 -9.12250078e-04]]
-
class
tinyms.layers.ReLU[source]¶ Rectified Linear Unit activation function.
Applies the rectified linear unit function element-wise.
\[\text{ReLU}(x) = (x)^+ = \max(0, x),\]It returns element-wise \(\max(0, x)\), specially, the neurons with the negative output will be suppressed and the active neurons will stay the same.
The picture about ReLU looks like this ReLU.
- Inputs:
input_data (Tensor) - The input of ReLU.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If dtype of input_data is not a number.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1, 2, -3, 2, -1]), mindspore.float16) >>> relu = nn.ReLU() >>> output = relu(input_x) >>> print(output) [0. 2. 0. 2. 0.]
-
class
tinyms.layers.ReLU6[source]¶ Compute ReLU6 activation function.
ReLU6 is similar to ReLU with a upper limit of 6, which if the inputs are greater than 6, the outputs will be suppressed to 6. It computes element-wise as
\[\min(\max(0, x), 6).\]The input is a Tensor of any valid shape.
- Inputs:
input_data (Tensor) - The input of ReLU6 with data type of float16 or float32.
- Outputs:
Tensor, which has the same type as input_data.
- Raises
TypeError – If dtype of input_data is neither float16 nor float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> relu6 = nn.ReLU6() >>> output = relu6(input_x) >>> print(output) [0. 0. 0. 2. 1.]
-
class
tinyms.layers.Tanh[source]¶ Tanh activation function.
Applies the Tanh function element-wise, returns a new tensor with the hyperbolic tangent of the elements of input, The input is a Tensor with any valid shape.
Tanh function is defined as:
\[tanh(x_i) = \frac{\exp(x_i) - \exp(-x_i)}{\exp(x_i) + \exp(-x_i)} = \frac{\exp(2x_i) - 1}{\exp(2x_i) + 1},\]where \(x_i\) is an element of the input Tensor.
- Inputs:
input_data (Tensor) - The input of Tanh with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If dtype of input_data is neither float16 nor float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([1, 2, 3, 2, 1]), mindspore.float16) >>> tanh = nn.Tanh() >>> output = tanh(input_x) >>> print(output) [0.7617 0.964 0.995 0.964 0.7617]
-
class
tinyms.layers.GELU[source]¶ Gaussian error linear unit activation function.
Applies GELU function to each element of the input. The input is a Tensor with any valid shape.
GELU is defined as:
\[GELU(x_i) = x_i*P(X < x_i),\]where \(P\) is the cumulative distribution function of standard Gaussian distribution and \(x_i\) is the element of the input.
The picture about GELU looks like this GELU.
- Inputs:
input_data (Tensor) - The input of GELU with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If dtype of input_data is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> gelu = nn.GELU() >>> output = gelu(input_x) >>> print(output) [[-1.5880802e-01 3.9999299e+00 -3.1077917e-21] [ 1.9545976e+00 -2.2918017e-07 9.0000000e+00]]
-
class
tinyms.layers.FastGelu[source]¶ Fast Gaussian error linear unit activation function.
Applies FastGelu function to each element of the input. The input is a Tensor with any valid shape.
FastGelu is defined as:
\[FastGelu(x_i) = \frac {x_i} {1 + \exp(-1.702 * \left| x_i \right|)} * \exp(0.851 * (x_i - \left| x_i \right|))\]where \(x_i\) is the element of the input.
- Inputs:
input_data (Tensor) - The input of FastGelu with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If dtype of input_data is neither float16 nor float32.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> fast_gelu = nn.FastGelu() >>> output = fast_gelu(input_x) >>> print(output) [[-1.5420423e-01 3.9955850e+00 -9.7664279e-06] [ 1.9356586e+00 -1.0070159e-03 8.9999981e+00]]
-
class
tinyms.layers.Sigmoid[source]¶ Sigmoid activation function.
Applies sigmoid-type activation element-wise.
Sigmoid function is defined as:
\[\text{sigmoid}(x_i) = \frac{1}{1 + \exp(-x_i)},\]where \(x_i\) is the element of the input.
The picture about Sigmoid looks like this Sigmoid.
- Inputs:
input_data (Tensor) - The input of Sigmoid with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If dtype of input_data is neither float16 nor float32.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> sigmoid = nn.Sigmoid() >>> output = sigmoid(input_x) >>> print(output) [0.2688 0.11914 0.5 0.881 0.7305 ]
-
class
tinyms.layers.PReLU(channel=1, w=0.25)[source]¶ PReLU activation function.
Applies the PReLU function element-wise.
PReLU is defined as:
\[prelu(x_i)= \max(0, x_i) + w * \min(0, x_i),\]where \(x_i\) is an element of an channel of the input.
Here \(w\) is a learnable parameter with a default initial value 0.25. Parameter \(w\) has dimensionality of the argument channel. If called without argument channel, a single parameter \(w\) will be shared across all channels.
The picture about PReLU looks like this PReLU.
- Parameters
- Inputs:
input_data (Tensor) - The input of PReLU with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If channel is not an int.
TypeError – If w is not one of float, list, Tensor.
TypeError – If dtype of input_data is neither float16 nor float32.
ValueError – If channel is less than 1.
ValueError – If length of shape of input_data is equal to 1.
- Supported Platforms:
Ascend
Examples
>>> input_x = Tensor(np.array([[[[0.1, 0.6], [0.9, 0.9]]]]), mindspore.float32) >>> prelu = nn.PReLU() >>> output = prelu(input_x) >>> print(output) [[[[0.1 0.6] [0.9 0.9]]]]
-
tinyms.layers.get_activation(name)[source]¶ Gets the activation function.
- Parameters
name (str) – The name of the activation function.
- Returns
Function, the activation function.
- Supported Platforms:
AscendGPUCPU
Examples
>>> sigmoid = nn.get_activation('sigmoid')
-
class
tinyms.layers.LeakyReLU(alpha=0.2)[source]¶ Leaky ReLU activation function.
LeakyReLU is similar to ReLU, but LeakyReLU has a slope that makes it not equal to 0 at x < 0. The activation function is defined as:
\[\text{leaky_relu}(x) = \begin{cases}x, &\text{if } x \geq 0; \cr \text{alpha} * x, &\text{otherwise.}\end{cases}\]See https://ai.stanford.edu/~amaas/papers/relu_hybrid_icml2013_final.pdf
- Inputs:
input_x (Tensor) - The input of LeakyReLU.
- Outputs:
Tensor, has the same type and shape as the input_x.
- Raises
TypeError – If alpha is not a float or an int.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([[-1.0, 4.0, -8.0], [2.0, -5.0, 9.0]]), mindspore.float32) >>> leaky_relu = nn.LeakyReLU() >>> output = leaky_relu(input_x) >>> print(output) [[-0.2 4. -1.6] [ 2. -1. 9. ]]
-
class
tinyms.layers.HSigmoid[source]¶ Hard sigmoid activation function.
Applies hard sigmoid activation element-wise. The input is a Tensor with any valid shape.
Hard sigmoid is defined as:
\[\text{hsigmoid}(x_{i}) = max(0, min(1, \frac{x_{i} + 3}{6})),\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Inputs:
input_data (Tensor) - The input of HSigmoid, data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If dtype of input_data is neither float16 nor float32.
- Supported Platforms:
GPUCPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> hsigmoid = nn.HSigmoid() >>> result = hsigmoid(input_x) >>> print(result) [0.3333 0.1666 0.5 0.833 0.6665]
-
class
tinyms.layers.HSwish[source]¶ Hard swish activation function.
Applies hswish-type activation element-wise. The input is a Tensor with any valid shape.
Hard swish is defined as:
\[\text{hswish}(x_{i}) = x_{i} * \frac{ReLU6(x_{i} + 3)}{6},\]where \(x_{i}\) is the \(i\)-th slice in the given dimension of the input Tensor.
- Inputs:
input_data (Tensor) - The input of HSwish, data type must be float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If dtype of input_data is neither float16 nor float32.
- Supported Platforms:
GPUCPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float16) >>> hswish = nn.HSwish() >>> result = hswish(input_x) >>> print(result) [-0.3333 -0.3333 0 1.666 0.6665]
-
class
tinyms.layers.ELU(alpha=1.0)[source]¶ Exponential Linear Uint activation function.
Applies the exponential linear unit function element-wise. The activation function is defined as:
\[E_{i} = \begin{cases} x, &\text{if } x \geq 0; \cr \text{alpha} * (\exp(x_i) - 1), &\text{otherwise.} \end{cases}\]The picture about ELU looks like this ELU.
- Parameters
alpha (float) – The coefficient of negative factor whose type is float. Default: 1.0.
- Inputs:
input_data (Tensor) - The input of ELU with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If alpha is not a float.
TypeError – If dtype of input_data is neither float16 nor float32.
ValueError – If alpha is not equal to 1.0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input_x = Tensor(np.array([-1, -2, 0, 2, 1]), mindspore.float32) >>> elu = nn.ELU() >>> result = elu(input_x) >>> print(result) [-0.63212055 -0.86466473 0. 2. 1.]
-
class
tinyms.layers.LogSigmoid[source]¶ Logsigmoid activation function.
Applies logsigmoid activation element-wise. The input is a Tensor with any valid shape.
Logsigmoid is defined as:
\[\text{logsigmoid}(x_{i}) = log(\frac{1}{1 + \exp(-x_i)}),\]where \(x_{i}\) is the element of the input.
- Inputs:
input_data (Tensor) - The input of LogSigmoid with data type of float16 or float32.
- Outputs:
Tensor, with the same type and shape as the input_data.
- Raises
TypeError – If dtype of input_data is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.LogSigmoid() >>> input_x = Tensor(np.array([1.0, 2.0, 3.0]), mindspore.float32) >>> output = net(input_x) >>> print(output) [-0.31326166 -0.12692806 -0.04858734]
-
class
tinyms.layers.BatchNorm1d(num_features, eps=1e-05, momentum=0.9, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=None)[source]¶ Batch normalization layer over a 2D input.
Batch Normalization is widely used in convolutional networks. This layer applies Batch Normalization over a 2D input (a mini-batch of 1D inputs) to reduce internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]Note
The implementation of BatchNorm is different in graph mode and pynative mode, therefore the mode is not recommended to be changed after net was initialized.
- Parameters
num_features (int) – C from an expected input of size (N, C).
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.9.
affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.
moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.
moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.
use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. If None, the training process will use the mean and variance of current batch data and track the running mean and variance, the evaluation process will use the running mean and variance. Default: None.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in})\).
- Outputs:
Tensor, the normalized, scaled, offset tensor, of shape \((N, C_{out})\).
- Supported Platforms:
Ascend
- Raises
TypeError – If num_features is not an int.
TypeError – If eps is not a float.
ValueError – If num_features is less than 1.
ValueError – If momentum is not in range [0, 1].
Examples
>>> net = nn.BatchNorm1d(num_features=4) >>> np.random.seed(0) >>> input = Tensor(np.random.randint(0, 255, [2, 4]), mindspore.float32) >>> output = net(input) >>> print(output) [[171.99915 46.999763 116.99941 191.99904 ] [ 66.999664 250.99875 194.99902 102.99948 ]]
-
class
tinyms.layers.BatchNorm2d(num_features, eps=1e-05, momentum=0.9, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=None, data_format='NCHW')[source]¶ Batch normalization layer over a 4D input.
Batch Normalization is widely used in convolutional networks. This layer applies Batch Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) to avoid internal covariate shift as described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]Note
The implementation of BatchNorm is different in graph mode and pynative mode, therefore that mode can not be changed after net was initialized. Note that the formula for updating the running_mean and running_var is \(\hat{x}_\text{new} = (1 - \text{momentum}) \times x_t + \text{momentum} \times \hat{x}\), where \(\hat{x}\) is the estimated statistic and \(x_t\) is the new observed value.
- Parameters
num_features (int) – C from an expected input of size (N, C, H, W).
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.9.
affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.
moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.
moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.
use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. If None, the training process will use the mean and variance of current batch data and track the running mean and variance, the evaluation process will use the running mean and variance. Default: None.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, the normalized, scaled, offset tensor, of shape \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If num_features is not an int.
TypeError – If eps is not a float.
ValueError – If num_features is less than 1.
ValueError – If momentum is not in range [0, 1].
ValueError – If data_format is neither ‘NHWC’ not ‘NCHW’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.BatchNorm2d(num_features=3) >>> np.random.seed(0) >>> input = Tensor(np.random.randint(0, 255, [1, 3, 2, 2]), mindspore.float32) >>> output = net(input) >>> print(output) [[[[171.99915 46.999763 ] [116.99941 191.99904 ]] [[ 66.999664 250.99875 ] [194.99902 102.99948 ]] [[ 8.999955 210.99895 ] [ 20.999895 241.9988 ]]]]
-
class
tinyms.layers.BatchNorm3d(num_features, eps=1e-05, momentum=0.9, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=None, data_format='NCDHW')[source]¶ Batch normalization layer over a 5D input.
Batch Normalization is widely used in convolutional networks. This layer applies Batch Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension) to avoid internal covariate shift.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]Note
The implementation of BatchNorm is different in graph mode and pynative mode, therefore that mode can not be changed after net was initialized. Note that the formula for updating the running_mean and running_var is \(\hat{x}_\text{new} = (1 - \text{momentum}) \times x_t + \text{momentum} \times \hat{x}\), where \(\hat{x}\) is the estimated statistic and \(x_t\) is the new observed value.
- Parameters
num_features (int) – C from an expected input of size (N, C, D, H, W).
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.9.
affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.
moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.
moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.
use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. If None, the training process will use the mean and variance of current batch data and track the running mean and variance, the evaluation process will use the running mean and variance. Default: None.
data_format (str) – The optional value for data format is ‘NCDHW’. Default: ‘NCDHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, D_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, the normalized, scaled, offset tensor, of shape \((N, C_{out}, D_{out},H_{out}, W_{out})\).
- Raises
TypeError – If num_features is not an int.
TypeError – If eps is not a float.
ValueError – If num_features is less than 1.
ValueError – If momentum is not in range [0, 1].
ValueError – If data_format is not ‘NCDHW’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.BatchNorm3d(num_features=3) >>> np.random.seed(0) >>> input = Tensor(np.random.randint(0, 255, [16, 3, 10, 32, 32]), mindspore.float32) >>> output = net(input) >>> print(output.shape) (16, 3, 10, 32, 32)
-
class
tinyms.layers.LayerNorm(normalized_shape, begin_norm_axis=-1, begin_params_axis=-1, gamma_init='ones', beta_init='zeros', epsilon=1e-07)[source]¶ Applies Layer Normalization over a mini-batch of inputs.
Layer normalization is widely used in recurrent neural networks. It applies normalization on a mini-batch of inputs for each single training case as described in the paper Layer Normalization. Unlike batch normalization, layer normalization performs exactly the same computation at training and testing time. It can be described using the following formula. It is applied across all channels and pixel but only one batch size.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]- Parameters
normalized_shape (Union(tuple[int], list[int]) – The normalization is performed over axis begin_norm_axis … R - 1.
begin_norm_axis (int) – The first normalization dimension: normalization will be performed along dimensions begin_norm_axis: rank(inputs), the value should be in [-1, rank(input)). Default: -1.
begin_params_axis (int) – The first parameter(beta, gamma)dimension: scale and centering parameters will have dimensions begin_params_axis: rank(inputs) and will be broadcast with the normalized inputs accordingly, the value should be in [-1, rank(input)). Default: -1.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
epsilon (float) – A value added to the denominator for numerical stability. Default: 1e-7.
- Inputs:
input_x (Tensor) - The shape of ‘input_x’ is \((x_1, x_2, ..., x_R)\), and input_shape[begin_norm_axis:] is equal to normalized_shape.
- Outputs:
Tensor, the normalized and scaled offset tensor, has the same shape and data type as the input_x.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> x = Tensor(np.ones([20, 5, 10, 10]), mindspore.float32) >>> shape1 = x.shape[1:] >>> m = nn.LayerNorm(shape1, begin_norm_axis=1, begin_params_axis=1) >>> output = m(x).shape >>> print(output) (20, 5, 10, 10)
-
class
tinyms.layers.GroupNorm(num_groups, num_channels, eps=1e-05, affine=True, gamma_init='ones', beta_init='zeros')[source]¶ Group Normalization over a mini-batch of inputs.
Group normalization is widely used in recurrent neural networks. It applies normalization on a mini-batch of inputs for each single training case as described in the paper Group Normalization. Group normalization divides the channels into groups and computes within each group the mean and variance for normalization, and it performs very stable over a wide range of batch size. It can be described using the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]- Parameters
num_groups (int) – The number of groups to be divided along the channel dimension.
num_channels (int) – The number of channels per group.
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
affine (bool) – A bool value, this layer will have learnable affine parameters when set to true. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’. If gamma_init is a Tensor, the shape must be [num_channels].
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’. If beta_init is a Tensor, the shape must be [num_channels].
- Inputs:
input_x (Tensor) - The input feature with shape [N, C, H, W].
- Outputs:
Tensor, the normalized and scaled offset tensor, has the same shape and data type as the input_x.
- Raises
TypeError – If num_groups or num_channels is not an int.
TypeError – If eps is not a float.
TypeError – If affine is not a bool.
ValueError – If num_groups or num_channels is less than 1.
ValueError – If num_channels is not divided by num_groups.
- Supported Platforms:
AscendGPUCPU
Examples
>>> goup_norm_op = nn.GroupNorm(2, 2) >>> x = Tensor(np.ones([1, 2, 4, 4], np.float32)) >>> output = goup_norm_op(x) >>> print(output) [[[[0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.]] [[0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.] [0. 0. 0. 0.]]]]
-
class
tinyms.layers.GlobalBatchNorm(**kwargs)[source]¶ Global normalization layer over a N-dimension input.
Global Normalization is cross device synchronized batch normalization. The implementation of Batch Normalization only normalizes the data within each device. Global normalization will normalize the input within the group. It has been described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]Note
Currently, GlobalBatchNorm only supports 2D and 4D inputs.
- Parameters
num_features (int) – C from an expected input of size (N, C, H, W).
device_num_each_group (int) – The number of devices in each group. Default: 2.
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.9.
affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. If None, training process will use the mean and variance of current batch data and track the running mean and variance, eval process will use the running mean and variance. Default: None.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, the normalized, scaled, offset tensor, of shape \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If num_features or device_num_each_group is not an int.
TypeError – If eps is not a float.
ValueError – If num_features is less than 1.
ValueError – If momentum is not in range [0, 1].
ValueError – If device_num_each_group is less than 2.
- Supported Platforms:
Ascend
Examples
>>> # This example should be run with multiple processes. >>> # Please refer to the tutorial > Distributed Training on mindspore.cn. >>> import numpy as np >>> from mindspore.communication import init >>> from mindspore import context >>> from mindspore.context import ParallelMode >>> from mindspore import nn, Tensor >>> from mindspore.common import dtype as mstype >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> context.reset_auto_parallel_context() >>> context.set_auto_parallel_context(parallel_mode=ParallelMode.DATA_PARALLEL) >>> np.random.seed(0) >>> global_bn_op = nn.GlobalBatchNorm(num_features=3, device_num_each_group=2) >>> input = Tensor(np.random.randint(0, 255, [1, 3, 2, 2]), mstype.float32) >>> output = global_bn_op(input) >>> print(output) [[[[171.99915 46.999763] [116.99941 191.99904 ]] [[ 66.999664 250.99875 ] [194.99902 102.99948 ]] [[ 8.999955 210.99895 ] [ 20.9999895 241.9988 ]]]]
-
class
tinyms.layers.SyncBatchNorm(num_features, eps=1e-05, momentum=0.9, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=None, process_groups=None)[source]¶ Sync Batch normalization layer over a N-dimension input.
Sync Batch Normalization is cross device synchronized batch normalization. The implementation of Batch Normalization only normalizes the data within each device. Sync Batch normalization will normalize the input within the group. It has been described in the paper Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]Note
Currently, SyncBatchNorm only supports 2D and 4D inputs.
- Parameters
num_features (int) – C from an expected input of size (N, C, H, W).
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.9.
affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘zeros’.
moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, ‘xavier_uniform’, ‘he_uniform’, etc. Default: ‘ones’.
use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. If None, training process will use the mean and variance of current batch data and track the running mean and variance, eval process will use the running mean and variance. Default: None.
process_groups (list) – A list to divide devices into different sync groups, containing N subtraction lists. Each subtraction list contains int numbers identifying rank ids which need to be synchronized in the same group. All int values must be in [0, rank_size) and different from each other. Default: None, indicating synchronization across all devices.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor, the normalized, scaled, offset tensor, of shape \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If num_features is not an int.
TypeError – If eps is not a float.
TypeError – If process_groups is not a list.
ValueError – If num_features is less than 1.
ValueError – If momentum is not in range [0, 1].
ValueError – If rank_id in process_groups is not in range [0, rank_size).
- Supported Platforms:
Ascend
Examples
>>> # This example should be run with multiple processes. >>> # Please refer to the tutorial > Distributed Training on mindspore.cn. >>> import numpy as np >>> from mindspore.communication import init >>> from mindspore import context >>> from mindspore.context import ParallelMode >>> from mindspore import nn, Tensor >>> from mindspore.common import dtype as mstype >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> context.reset_auto_parallel_context() >>> context.set_auto_parallel_context(parallel_mode=ParallelMode.DATA_PARALLEL) >>> np.random.seed(0) >>> sync_bn_op = nn.SyncBatchNorm(num_features=3, process_groups=[[0, 1], [2, 3]]) >>> input = Tensor(np.random.randint(0, 255, [1, 3, 2, 2]), mstype.float32) >>> output = sync_bn_op(input) >>> print(output) [[[[171.99915 46.999763] [116.99941 191.99904 ]] [[ 66.999664 250.99875 ] [194.99902 102.99948 ]] [[ 8.999955 210.99895 ] [ 20.9999895 241.9988 ]]]]
-
class
tinyms.layers.InstanceNorm2d(num_features, eps=1e-05, momentum=0.1, affine=True, gamma_init='ones', beta_init='zeros', moving_mean_init='zeros', moving_var_init='ones', use_batch_statistics=True)[source]¶ Instance normalization layer over a 4D input.
This layer applies Instance Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) as described in the paper Instance Normalization: The Missing Ingredient for Fast Stylization. It rescales and recenters the feature using a mini-batch of data and the learned parameters which can be described in the following formula.
\[y = \frac{x - \mathrm{E}[x]}{\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta\]gamma and beta are learnable parameter vectors of size num_features if affine is True. The standard-deviation is calculated via the biased estimator.
By default, this layer uses instance statistics computed from input data in both training and evaluation modes.
If use_batch_statistics is set to True, it means training phases, and this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default momentum of 0.1.
InstanceNorm2d and BatchNorm2d are very similar, but have some differences. InstanceNorm2d is applied on each channel of channeled data like RGB images, but BatchNorm2d is usually applied on each batch of batched data.
Note
Note that the formula for updating the running_mean and running_var is \(\hat{x}_\text{new} = (1 - \text{momentum}) \times x_t + \text{momentum} \times \hat{x}\), where \(\hat{x}\) is the estimated statistic and \(x_t\) is the new observed value.
- Parameters
num_features (int) – C from an expected input of size (N, C, H, W).
eps (float) – A value added to the denominator for numerical stability. Default: 1e-5.
momentum (float) – A floating hyperparameter of the momentum for the running_mean and running_var computation. Default: 0.1.
affine (bool) – A bool value. When set to True, gamma and beta can be learned. Default: True.
gamma_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the gamma weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.
beta_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the beta weight. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.
moving_mean_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving mean. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘zeros’.
moving_var_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the moving variance. The values of str refer to the function initializer including ‘zeros’, ‘ones’, etc. Default: ‘ones’.
use_batch_statistics (bool) – If true, use the mean value and variance value of current batch data. If false, use the mean value and variance value of specified value. Default: True.
- Inputs:
input (Tensor) - Tensor of shape \((N, C, H, W)\). Data type: float16 or float32.
- Outputs:
Tensor, the normalized, scaled, offset tensor, of shape \((N, C, H, W)\). Same type and shape as the input_x.
- Supported Platforms:
GPU
- Raises
TypeError – If num_features is not an int.
TypeError – If eps is not a float.
TypeError – If momentum is not a float.
TypeError – If affine is not a bool.
TypeError – If the type of gamma_init/beta_init/moving_mean_init/moving_var_init is not same, or if the initialized element type is not float32.
ValueError – If num_features is less than 1.
ValueError – If momentum is not in range [0, 1].
KeyError – If any of gamma_init/beta_init/moving_mean_init/moving_var_init is str and the homonymous class inheriting from Initializer not exists.
Examples
>>> net = nn.InstanceNorm2d(3) >>> np.random.seed(0) >>> input = Tensor(np.random.randint(0, 255, [2, 3, 2, 2]), mindspore.float32) >>> output = net(input) >>> print(output.shape) (2, 3, 2, 2)
-
class
tinyms.layers.Conv2d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros', data_format='NCHW')[source]¶ 2D convolution layer.
Applies a 2D convolution over an input tensor which is typically of shape \((N, C_{in}, H_{in}, W_{in})\), where \(N\) is batch size, \(C_{in}\) is channel number, and \(H_{in}, W_{in})\) are height and width. For each batch of shape \((C_{in}, H_{in}, W_{in})\), the formula is defined as:
\[out_j = \sum_{i=0}^{C_{in} - 1} ccor(W_{ij}, X_i) + b_j,\]where \(ccor\) is the cross-correlation operator, \(C_{in}\) is the input channel number, \(j\) ranges from \(0\) to \(C_{out} - 1\), \(W_{ij}\) corresponds to the \(i\)-th channel of the \(j\)-th filter and \(out_{j}\) corresponds to the \(j\)-th channel of the output. \(W_{ij}\) is a slice of kernel and it has shape \((\text{ks_h}, \text{ks_w})\), where \(\text{ks_h}\) and \(\text{ks_w}\) are the height and width of the convolution kernel. The full kernel has shape \((C_{out}, C_{in} // \text{group}, \text{ks_h}, \text{ks_w})\), where group is the group number to split the input in the channel dimension.
If the ‘pad_mode’ is set to be “valid”, the output height and width will be \(\left \lfloor{1 + \frac{H_{in} + 2 \times \text{padding} - \text{ks_h} - (\text{ks_h} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) and \(\left \lfloor{1 + \frac{W_{in} + 2 \times \text{padding} - \text{ks_w} - (\text{ks_w} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) respectively.
The first introduction can be found in paper Gradient Based Learning Applied to Document Recognition.
- Parameters
in_channels (int) – The number of input channel \(C_{in}\).
out_channels (int) – The number of output channel \(C_{out}\).
kernel_size (Union[int, tuple[int]]) – The data type is int or a tuple of 2 integers. Specifies the height and width of the 2D convolution window. Single int means the value is for both the height and the width of the kernel. A tuple of 2 ints means the first value is for the height and the other is for the width of the kernel.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
Specifies padding mode. The optional values are “same”, “valid”, “pad”. Default: “same”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side. If this mode is set, padding must be 0.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded. If this mode is set, padding must be 0.
pad: Implicit paddings on both sides of the input. The number of padding will be padded to the input Tensor borders. padding must be greater than or equal to 0.
padding (Union[int, tuple[int]]) – Implicit paddings on both sides of the input. If padding is one integer, the paddings of top, bottom, left and right are the same, equal to padding. If padding is a tuple with four integers, the paddings of top, bottom, left and right will be equal to padding[0], padding[1], padding[2], and padding[3] accordingly. Default: 0.
dilation (Union[int, tuple[int]]) – The data type is int or a tuple of 2 integers. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater or equal to 1 and bounded by the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_ channels and out_channels must be divisible by the number of groups. If the group is equal to in_channels and out_channels, this 2D convolution layer also can be called 2D depthwise convolution layer. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\) or \((N, H_{in}, W_{in}, C_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\) or \((N, H_{out}, W_{out}, C_{out})\).
- Raises
TypeError – If in_channels, out_channels or group is not an int.
TypeError – If kernel_size, stride, padding or dilation is neither an int not a tuple.
ValueError – If in_channels, out_channels, kernel_size, stride or dilation is less than 1.
ValueError – If padding is less than 0.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
ValueError – If padding is a tuple whose length is not equal to 4.
ValueError – If pad_mode is not equal to ‘pad’ and padding is not equal to (0, 0, 0, 0).
ValueError – If data_format is neither ‘NCHW’ not ‘NHWC’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.Conv2d(120, 240, 4, has_bias=False, weight_init='normal') >>> input = Tensor(np.ones([1, 120, 1024, 640]), mindspore.float32) >>> output = net(input).shape >>> print(output) (1, 240, 1024, 640)
-
class
tinyms.layers.Conv2dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')[source]¶ 2D transposed convolution layer.
Compute a 2D transposed convolution, which is also known as a deconvolution (although it is not an actual deconvolution).
Input is typically of shape \((N, C, H, W)\), where \(N\) is batch size and \(C\) is channel number.
If the ‘pad_mode’ is set to be “pad”, the height and width of output are defined as:
\[ \begin{align}\begin{aligned}H_{out} = (H_{in} - 1) \times \text{stride} - 2 \times \text{padding} + \text{dilation} \times (\text{ks_h} - 1) + 1\\W_{out} = (W_{in} - 1) \times \text{stride} - 2 \times \text{padding} + \text{dilation} \times (\text{ks_w} - 1) + 1\end{aligned}\end{align} \]where \(\text{ks_h}\) is the height of the convolution kernel and \(\text{ks_w}\) is the width of the convolution kernel.
- Parameters
in_channels (int) – The number of channels in the input space.
out_channels (int) – The number of channels in the output space.
kernel_size (Union[int, tuple]) – int or a tuple of 2 integers, which specifies the height and width of the 2D convolution window. Single int means the value is for both the height and the width of the kernel. A tuple of 2 ints means the first value is for the height and the other is for the width of the kernel.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Its value must be equal to or greater than 1. Default: 1.
pad_mode (str) –
Select the mode of the pad. The optional values are “pad”, “same”, “valid”. Default: “same”.
pad: Implicit paddings on both sides of the input.
same: Adopted the way of completion.
valid: Adopted the way of discarding.
padding (Union[int, tuple[int]]) – Implicit paddings on both sides of the input. If padding is one integer, the paddings of top, bottom, left and right are the same, equal to padding. If padding is a tuple with four integers, the paddings of top, bottom, left and right will be equal to padding[0], padding[1], padding[2], and padding[3] accordingly. Default: 0.
dilation (Union[int, tuple[int]]) – The data type is int or a tuple of 2 integers. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater than or equal to 1 and bounded by the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_channels and out_channels must be divisible by the number of groups. This does not support for Davinci devices when group > 1. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If in_channels, out_channels or group is not an int.
TypeError – If kernel_size, stride, padding or dilation is neither an int not a tuple.
ValueError – If in_channels, out_channels, kernel_size, stride or dilation is less than 1.
ValueError – If padding is less than 0.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
ValueError – If padding is a tuple whose length is not equal to 4.
ValueError – If pad_mode is not equal to ‘pad’ and padding is not equal to (0, 0, 0, 0).
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Conv2dTranspose(3, 64, 4, has_bias=False, weight_init='normal', pad_mode='pad') >>> input = Tensor(np.ones([1, 3, 16, 50]), mindspore.float32) >>> output = net(input).shape >>> print(output) (1, 64, 19, 53)
-
class
tinyms.layers.Conv1d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')[source]¶ 1D convolution layer.
Applies a 1D convolution over an input tensor which is typically of shape \((N, C_{in}, W_{in})\), where \(N\) is batch size and \(C_{in}\) is channel number. For each batch of shape \((C_{in}, W_{in})\), the formula is defined as:
\[out_j = \sum_{i=0}^{C_{in} - 1} ccor(W_{ij}, X_i) + b_j,\]where \(ccor\) is the cross correlation operator, \(C_{in}\) is the input channel number, \(j\) ranges from \(0\) to \(C_{out} - 1\), \(W_{ij}\) corresponds to the \(i\)-th channel of the \(j\)-th filter and \(out_{j}\) corresponds to the \(j\)-th channel of the output. \(W_{ij}\) is a slice of kernel and it has shape \((\text{ks_w})\), where \(\text{ks_w}\) is the width of the convolution kernel. The full kernel has shape \((C_{out}, C_{in} // \text{group}, \text{ks_w})\), where group is the group number to split the input in the channel dimension.
If the ‘pad_mode’ is set to be “valid”, the output width will be \(\left \lfloor{1 + \frac{W_{in} + 2 \times \text{padding} - \text{ks_w} - (\text{ks_w} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) respectively.
The first introduction of convolution layer can be found in paper Gradient Based Learning Applied to Document Recognition.
- Parameters
in_channels (int) – The number of input channel \(C_{in}\).
out_channels (int) – The number of output channel \(C_{out}\).
kernel_size (int) – The data type is int. Specifies the width of the 1D convolution window.
stride (int) – The distance of kernel moving, an int number that represents the width of movement. Default: 1.
pad_mode (str) –
Specifies padding mode. The optional values are “same”, “valid”, “pad”. Default: “same”.
same: Adopts the way of completion. The output width will be the same as the input. The total number of padding will be calculated in the horizontal direction and evenly distributed to left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side. If this mode is set, padding must be 0.
valid: Adopts the way of discarding. The possible largest width of the output will be returned without padding. Extra pixels will be discarded. If this mode is set, padding must be 0.
pad: Implicit paddings on both sides of the input. The number of padding will be padded to the input Tensor borders. padding must be greater than or equal to 0.
padding (int) – Implicit paddings on both sides of the input. Default: 0.
dilation (int) – The data type is int. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater or equal to 1 and bounded by the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_ channels and out_channels must be divisible by the number of groups. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – An initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, W_{out})\).
- Raises
TypeError – If in_channels, out_channels, kernel_size, stride, padding or dilation is not an int.
ValueError – If in_channels, out_channels, kernel_size, stride or dilation is less than 1.
ValueError – If padding is less than 0.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.Conv1d(120, 240, 4, has_bias=False, weight_init='normal') >>> input = Tensor(np.ones([1, 120, 640]), mindspore.float32) >>> output = net(input).shape >>> print(output) (1, 240, 640)
-
class
tinyms.layers.Conv1dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')[source]¶ 1D transposed convolution layer.
Compute a 1D transposed convolution, which is also known as a deconvolution (although it is not an actual deconvolution).
Input is typically of shape \((N, C, W)\), where \(N\) is batch size and \(C\) is channel number.
If the ‘pad_mode’ is set to be “pad”, the width of output is defined as:
\[W_{out} = (W_{in} - 1) \times \text{stride} - 2 \times \text{padding} + \text{dilation} \times (\text{ks_w} - 1) + 1\]where \(\text{ks_w}\) is the width of the convolution kernel.
- Parameters
in_channels (int) – The number of channels in the input space.
out_channels (int) – The number of channels in the output space.
kernel_size (int) – int, which specifies the width of the 1D convolution window.
stride (int) – The distance of kernel moving, an int number that represents the width of movement. Default: 1.
pad_mode (str) –
Select the mode of the pad. The optional values are “pad”, “same”, “valid”. Default: “same”.
pad: Implicit paddings on both sides of the input.
same: Adopted the way of completion.
valid: Adopted the way of discarding.
padding (int) – Implicit paddings on both sides of the input. Default: 0.
dilation (int) – The data type is int. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater or equal to 1 and bounded by the width of the input. Default: 1.
group (int) – Splits filter into groups, in_channels and out_channels must be divisible by the number of groups. This is not support for Davinci devices when group > 1. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a numbers.Number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, W_{out})\).
- Raises
TypeError – If in_channels, out_channels, kernel_size, stride, padding or dilation is not an int.
ValueError – If in_channels, out_channels, kernel_size, stride or dilation is less than 1.
ValueError – If padding is less than 0.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Conv1dTranspose(3, 64, 4, has_bias=False, weight_init='normal', pad_mode='pad') >>> input = Tensor(np.ones([1, 3, 50]), mindspore.float32) >>> output = net(input).shape >>> print(output) (1, 64, 53)
-
class
tinyms.layers.Conv3d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros', data_format='NCDHW')[source]¶ 3D convolution layer.
Applies a 3D convolution over an input tensor which is typically of shape \((N, C_{in}, D_{in}, H_{in}, W_{in})\) and output shape \((N, C_{out}, D_{out}, H_{out}, W_{out})\). where \(N\) is batch size. \(C\) is channel number. the formula is defined as:
\[\operatorname{out}\left(N_{i}, C_{\text {out}_j}\right)=\operatorname{bias}\left(C_{\text {out}_j}\right)+ \sum_{k=0}^{C_{in}-1} ccor(\text {weight}\left(C_{\text {out}_j}, k\right), \operatorname{input}\left(N_{i}, k\right))\]where \(ccor\) is the cross-correlation operator.
If the ‘pad_mode’ is set to be “valid”, the output height and width will be \(\left \lfloor{1 + \frac{D_{in} + 2 \times \text{padding} - \text{ks_d} - (\text{ks_d} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) and \(\left \lfloor{1 + \frac{H_{in} + 2 \times \text{padding} - \text{ks_h} - (\text{ks_h} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) and \(\left \lfloor{1 + \frac{W_{in} + 2 \times \text{padding} - \text{ks_w} - (\text{ks_w} - 1) \times (\text{dilation} - 1) }{\text{stride}}} \right \rfloor\) respectively.
- Parameters
in_channels (int) – The number of input channel \(C_{in}\).
out_channels (int) – The number of output channel \(C_{out}\).
kernel_size (Union[int, tuple[int]]) – The data type is int or a tuple of 3 integers. Specifies the depth, height and width of the 3D convolution window. Single int means the value is for the depth, height and the width of the kernel. A tuple of 3 ints means the first value is for the depth, second value is for height and the other is for the width of the kernel.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the depth, height and width of movement are both strides, or a tuple of three int numbers that represent depth, height and width of movement respectively. Default: 1.
pad_mode (str) –
Specifies padding mode. The optional values are “same”, “valid”, “pad”. Default: “same”.
same: Adopts the way of completion. The depth, height and width of the output will be the same as the input. The total number of padding will be calculated in depth, horizontal and vertical directions and evenly distributed to head and tail, top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the tail, bottom and the right side. If this mode is set, padding must be 0.
valid: Adopts the way of discarding. The possible largest depth, height and width of output will be returned without padding. Extra pixels will be discarded. If this mode is set, padding must be 0.
pad: Implicit paddings on both sides of the input in depth, height, width. The number of padding will be padded to the input Tensor borders. padding must be greater than or equal to 0.
padding (Union(int, tuple[int])) – Implicit paddings on both sides of the input. The data type is int or a tuple of 6 integers. Default: 0. If padding is an integer, the paddings of head, tail, top, bottom, left and right are the same, equal to padding. If paddings is a tuple of six integers, the padding of head, tail, top, bottom, left and right equal to padding[0], padding[1], padding[2], padding[3], padding[4] and padding[5] correspondingly.
dilation (Union[int, tuple[int]]) – The data type is int or a tuple of 3 integers : math:(dilation_d, dilation_h, dilation_w). Currently, dilation on depth only supports the case of 1. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater or equal to 1 and bounded by the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_ channels and out_channels must be divisible by the number of groups. Default: 1. Only 1 is currently supported.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
data_format (str) – The optional value for data format. Currently only support “NCDHW”.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, D_{in}, H_{in}, W_{in})\). Currently input data type only support float16 and float32.
- Outputs:
Tensor, the value that applied 3D convolution. The shape is \((N, C_{out}, D_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If in_channels, out_channels or group is not an int.
TypeError – If kernel_size, stride, padding or dilation is neither an int nor a tuple of six.
ValueError – If out_channels, kernel_size, stride or dilation is less than 1.
ValueError – If padding is less than 0.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
ValueError – If padding is a tuple whose length is not equal to 6.
ValueError – If pad_mode is not equal to ‘pad’ and padding is not equal to (0, 0, 0, 0, 0, 0).
ValueError – If data_format is not ‘NCDHW’.
- Supported Platforms:
Ascend
Examples
>>> input = Tensor(np.ones([16, 3, 10, 32, 32]), mindspore.float32) >>> conv3d = nn.Conv3d(in_channels=3, out_channels=32, kernel_size=(4, 3, 3)) >>> output = conv3d(input) >>> print(output.shape) (16, 32, 10, 32, 32)
-
class
tinyms.layers.Conv3dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, output_padding=0, has_bias=False, weight_init='normal', bias_init='zeros', data_format='NCDHW')[source]¶ Compute a 3D transposed convolution, which is also known as a deconvolution (although it is not an actual deconvolution).
Input is typically of shape \((N, C, D, H, W)\), where \(N\) is batch size and \(C\) is channel number.
If the ‘pad_mode’ is set to be “pad”, the height and width of output are defined as:
\[ \begin{align}\begin{aligned}D_{out} = (D_{in} - 1) \times \text{stride_d} - 2 \times \text{padding_d} + \text{dilation_d} \times (\text{kernel_size_d} - 1) + \text{output_padding_d} + 1\\H_{out} = (H_{in} - 1) \times \text{stride_h} - 2 \times \text{padding_h} + \text{dilation_h} \times (\text{kernel_size_h} - 1) + \text{output_padding_h} + 1\\W_{out} = (W_{in} - 1) \times \text{stride_w} - 2 \times \text{padding_w} + \text{dilation_w} \times (\text{kernel_size_w} - 1) + \text{output_padding_w} + 1\end{aligned}\end{align} \]- Parameters
in_channels (int) – The number of input channel \(C_{in}\).
out_channels (int) – The number of output channel \(C_{out}\).
kernel_size (Union[int, tuple[int]]) – The kernel size of the 3D convolution.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the depth, height and width of movement are both strides, or a tuple of three int numbers that represent depth, height and width of movement respectively. Its value must be equal to or greater than 1. Default: 1.
pad_mode (str) –
Select the mode of the pad. The optional values are “pad”, “same”, “valid”. Default: “same”.
same: Adopts the way of completion. The depth, height and width of the output will be the same as the input. The total number of padding will be calculated in depth, horizontal and vertical directions and evenly distributed to head and tail, top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the tail, bottom and the right side. If this mode is set, padding and output_padding must be 0.
valid: Adopts the way of discarding. The possible largest depth, height and width of output will be returned without padding. Extra pixels will be discarded. If this mode is set, padding and output_padding must be 0.
pad: Implicit paddings on both sides of the input in depth, height, width. The number of pad will be padded to the input Tensor borders. padding must be greater than or equal to 0.
padding (Union(int, tuple[int])) – The pad value to be filled. Default: 0. If padding is an integer, the paddings of head, tail, top, bottom, left and right are the same, equal to padding. If padding is a tuple of six integers, the padding of head, tail, top, bottom, left and right equal to padding[0], padding[1], padding[2], padding[3], padding[4] and padding[5] correspondingly.
dilation (Union(int, tuple[int])) – The data type is int or a tuple of 3 integers : math:(dilation_d, dilation_h, dilation_w). Currently, dilation on depth only supports the case of 1. Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater or equal to 1 and bounded by the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_ channels and out_channels must be divisible by the number of groups. Default: 1. Only 1 is currently supported.
output_padding (Union(int, tuple[int])) – Add extra size to each dimension of the output. Default: 0. Must be greater than or equal to 0.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
data_format (str) – The optional value for data format. Currently only support ‘NCDHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, D_{in}, H_{in}, W_{in})\). Currently input data type only support float16 and float32.
- Outputs:
Tensor, the shape is \((N, C_{out}, D_{out}, H_{out}, W_{out})\).
- Supported Platforms:
Ascend
- Raises
TypeError – If in_channels, out_channels or group is not an int.
TypeError – If kernel_size, stride, padding , dilation or output_padding is neither an int not a tuple of three.
TypeError – If input data type is not float16 or float32.
ValueError – If in_channels, out_channels, kernel_size, stride or dilation is less than 1.
ValueError – If padding is less than 0.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
ValueError – If padding is a tuple whose length is not equal to 6.
ValueError – If pad_mode is not equal to ‘pad’ and padding is not equal to (0, 0, 0, 0, 0, 0).
ValueError – If data_format is not ‘NCDHW’.
Examples
>>> input = Tensor(np.ones([32, 16, 10, 32, 32]), mindspore.float32) >>> conv3d_transpose = nn.Conv3dTranspose(in_channels=16, out_channels=3, kernel_size=(4, 6, 2), pad_mode='pad') >>> output = conv3d_transpose(input) >>> print(output.shape) (32, 3, 13, 37, 33)
-
class
tinyms.layers.LSTM(input_size, hidden_size, num_layers=1, has_bias=True, batch_first=False, dropout=0, bidirectional=False)[source]¶ Stacked LSTM (Long Short-Term Memory) layers.
Apply LSTM layer to the input.
There are two pipelines connecting two consecutive cells in a LSTM model; one is cell state pipeline and the other is hidden state pipeline. Denote two consecutive time nodes as \(t-1\) and \(t\). Given an input \(x_t\) at time \(t\), an hidden state \(h_{t-1}\) and an cell state \(c_{t-1}\) of the layer at time \({t-1}\), the cell state and hidden state at time \(t\) is computed using an gating mechanism. Input gate \(i_t\) is designed to protect the cell from perturbation by irrelevant inputs. Forget gate \(f_t\) affords protection of the cell by forgetting some information in the past, which is stored in \(h_{t-1}\). Output gate \(o_t\) protects other units from perturbation by currently irrelevant memory contents. Candidate cell state \(\tilde{c}_t\) is calculated with the current input, on which the input gate will be applied. Finally, current cell state \(c_{t}\) and hidden state \(h_{t}\) are computed with the calculated gates and cell states. The complete formulation is as follows.
\[\begin{split}\begin{array}{ll} \\ i_t = \sigma(W_{ix} x_t + b_{ix} + W_{ih} h_{(t-1)} + b_{ih}) \\ f_t = \sigma(W_{fx} x_t + b_{fx} + W_{fh} h_{(t-1)} + b_{fh}) \\ \tilde{c}_t = \tanh(W_{cx} x_t + b_{cx} + W_{ch} h_{(t-1)} + b_{ch}) \\ o_t = \sigma(W_{ox} x_t + b_{ox} + W_{oh} h_{(t-1)} + b_{oh}) \\ c_t = f_t * c_{(t-1)} + i_t * \tilde{c}_t \\ h_t = o_t * \tanh(c_t) \\ \end{array}\end{split}\]Here \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product. \(W, b\) are learnable weights between the output and the input in the formula. For instance, \(W_{ix}, b_{ix}\) are the weight and bias used to transform from input \(x\) to \(i\). Details can be found in paper LONG SHORT-TERM MEMORY and Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling.
- Parameters
input_size (int) – Number of features of input.
hidden_size (int) – Number of features of hidden layer.
num_layers (int) – Number of layers of stacked LSTM . Default: 1.
has_bias (bool) – Whether the cell has bias b_ih and b_hh. Default: True.
batch_first (bool) – Specifies whether the first dimension of input is batch_size. Default: False.
dropout (float, int) – If not 0, append Dropout layer on the outputs of each LSTM layer except the last layer. Default 0. The range of dropout is [0.0, 1.0].
bidirectional (bool) – Specifies whether it is a bidirectional LSTM. Default: False.
- Inputs:
input (Tensor) - Tensor of shape (seq_len, batch_size, input_size) or (batch_size, seq_len, input_size).
hx (tuple) - A tuple of two Tensors (h_0, c_0) both of data type mindspore.float32 or mindspore.float16 and shape (num_directions * num_layers, batch_size, hidden_size). Data type of hx must be the same as input.
- Outputs:
Tuple, a tuple contains (output, (h_n, c_n)).
output (Tensor) - Tensor of shape (seq_len, batch_size, num_directions * hidden_size).
hx_n (tuple) - A tuple of two Tensor (h_n, c_n) both of shape (num_directions * num_layers, batch_size, hidden_size).
- Raises
TypeError – If input_size, hidden_size or num_layers is not an int.
TypeError – If has_bias, batch_first or bidirectional is not a bool.
TypeError – If dropout is neither a float nor an int.
ValueError – If dropout is not in range [0.0, 1.0].
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.LSTM(10, 16, 2, has_bias=True, batch_first=True, bidirectional=False) >>> input = Tensor(np.ones([3, 5, 10]).astype(np.float32)) >>> h0 = Tensor(np.ones([1 * 2, 3, 16]).astype(np.float32)) >>> c0 = Tensor(np.ones([1 * 2, 3, 16]).astype(np.float32)) >>> output, (hn, cn) = net(input, (h0, c0)) >>> print(output.shape) (3, 5, 16)
-
class
tinyms.layers.LSTMCell(input_size, hidden_size, has_bias=True, batch_first=False, dropout=0, bidirectional=False)[source]¶ LSTM (Long Short-Term Memory) layer.
Apply LSTM layer to the input.
There are two pipelines connecting two consecutive cells in a LSTM model; one is cell state pipeline and the other is hidden state pipeline. Denote two consecutive time nodes as \(t-1\) and \(t\). Given an input \(x_t\) at time \(t\), an hidden state \(h_{t-1}\) and an cell state \(c_{t-1}\) of the layer at time \({t-1}\), the cell state and hidden state at time \(t\) is computed using an gating mechanism. Input gate \(i_t\) is designed to protect the cell from perturbation by irrelevant inputs. Forget gate \(f_t\) affords protection of the cell by forgetting some information in the past, which is stored in \(h_{t-1}\). Output gate \(o_t\) protects other units from perturbation by currently irrelevant memory contents. Candidate cell state \(\tilde{c}_t\) is calculated with the current input, on which the input gate will be applied. Finally, current cell state \(c_{t}\) and hidden state \(h_{t}\) are computed with the calculated gates and cell states. The complete formulation is as follows.
\[\begin{split}\begin{array}{ll} \\ i_t = \sigma(W_{ix} x_t + b_{ix} + W_{ih} h_{(t-1)} + b_{ih}) \\ f_t = \sigma(W_{fx} x_t + b_{fx} + W_{fh} h_{(t-1)} + b_{fh}) \\ \tilde{c}_t = \tanh(W_{cx} x_t + b_{cx} + W_{ch} h_{(t-1)} + b_{ch}) \\ o_t = \sigma(W_{ox} x_t + b_{ox} + W_{oh} h_{(t-1)} + b_{oh}) \\ c_t = f_t * c_{(t-1)} + i_t * \tilde{c}_t \\ h_t = o_t * \tanh(c_t) \\ \end{array}\end{split}\]Here \(\sigma\) is the sigmoid function, and \(*\) is the Hadamard product. \(W, b\) are learnable weights between the output and the input in the formula. For instance, \(W_{ix}, b_{ix}\) are the weight and bias used to transform from input \(x\) to \(i\). Details can be found in paper LONG SHORT-TERM MEMORY and Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling.
LSTMCell is a single-layer RNN, you can achieve multi-layer RNN by stacking LSTMCell.
- Parameters
input_size (int) – Number of features of input.
hidden_size (int) – Number of features of hidden layer.
has_bias (bool) – Whether the cell has bias b_ih and b_hh. Default: True.
batch_first (bool) – Specifies whether the first dimension of input is batch_size. Default: False.
dropout (float, int) – If not 0, append Dropout layer on the outputs of each LSTM layer except the last layer. Default 0. The range of dropout is [0.0, 1.0].
bidirectional (bool) – Specifies whether this is a bidirectional LSTM. If set True, number of directions will be 2 otherwise number of directions is 1. Default: False.
- Inputs:
input (Tensor) - Tensor of shape (seq_len, batch_size, input_size).
h - data type mindspore.float32 or mindspore.float16 and shape (num_directions, batch_size, hidden_size).
c - data type mindspore.float32 or mindspore.float16 and shape (num_directions, batch_size, hidden_size). Data type of h’ and ‘c’ must be the same of `input.
w - data type mindspore.float32 or mindspore.float16 and shape (weight_size, 1, 1). The value of weight_size depends on input_size, hidden_size and bidirectional
- Outputs:
output, h_n, c_n, ‘reserve’, ‘state’.
output (Tensor) - Tensor of shape (seq_len, batch_size, num_directions * hidden_size).
h - A Tensor with shape (num_directions, batch_size, hidden_size).
c - A Tensor with shape (num_directions, batch_size, hidden_size).
reserve - reserved
state - reserved
- Raises
TypeError – If input_size or hidden_size or num_layers is not an int.
TypeError – If has_bias or batch_first or bidirectional is not a bool.
TypeError – If dropout is neither a float nor an int.
ValueError – If dropout is not in range [0.0, 1.0].
- Supported Platforms:
GPUCPU
Examples
>>> net = nn.LSTMCell(10, 12, has_bias=True, batch_first=True, bidirectional=False) >>> input = Tensor(np.ones([3, 5, 10]).astype(np.float32)) >>> h = Tensor(np.ones([1, 3, 12]).astype(np.float32)) >>> c = Tensor(np.ones([1, 3, 12]).astype(np.float32)) >>> w = Tensor(np.ones([1152, 1, 1]).astype(np.float32)) >>> output, h, c, _, _ = net(input, h, c, w) >>> print(output.shape) (3, 5, 12)
-
class
tinyms.layers.Dropout(keep_prob=0.5, dtype=mindspore.float32)[source]¶ Dropout layer for the input.
Randomly set some elements of the input tensor to zero with probability \(1 - keep\_prob\) during training using samples from a Bernoulli distribution.
The outputs are scaled by a factor of \(\frac{1}{keep\_prob}\) during training so that the output layer remains at a similar scale. During inference, this layer returns the same tensor as the input.
This technique is proposed in paper Dropout: A Simple Way to Prevent Neural Networks from Overfitting and proved to be effective to reduce over-fitting and prevents neurons from co-adaptation. See more details in Improving neural networks by preventing co-adaptation of feature detectors.
Note
Each channel will be zeroed out independently on every construct call.
- Parameters
keep_prob (float) – The keep rate, greater than 0 and less equal than 1. E.g. rate=0.9, dropping out 10% of input units. Default: 0.5.
dtype (
mindspore.dtype) – Data type of input. Default: mindspore.float32.
- Inputs:
input (Tensor) - The input of Dropout with data type of float16 or float32.
- Outputs:
Tensor, output tensor with the same shape as the input.
- Raises
TypeError – If keep_prob is not a float.
TypeError – If dtype of input is not neither float16 nor float32.
ValueError – If keep_prob is not in range (0, 1].
ValueError – If length of shape of input is less than 1.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.ones([2, 2, 3]), mindspore.float32) >>> net = nn.Dropout(keep_prob=0.8) >>> net.set_train() Dropout<keep_prob=0.8> >>> output = net(x) >>> print(output.shape) (2, 2, 3)
-
class
tinyms.layers.Flatten[source]¶ Flatten layer for the input.
Flattens a tensor without changing dimension of batch size on the 0-th axis.
- Inputs:
input (Tensor) - Tensor of shape \((N, \ldots)\) to be flattened.
- Outputs:
Tensor, the shape of the output tensor is \((N, X)\), where \(X\) is the product of the remaining dimensions.
- Raises
TypeError – If input is not a subclass of Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input = Tensor(np.array([[[1.2, 1.2], [2.1, 2.1]], [[2.2, 2.2], [3.2, 3.2]]]), mindspore.float32) >>> net = nn.Flatten() >>> output = net(input) >>> print(output) [[1.2 1.2 2.1 2.1] [2.2 2.2 3.2 3.2]]
-
class
tinyms.layers.Dense(in_channels, out_channels, weight_init='normal', bias_init='zeros', has_bias=True, activation=None)[source]¶ The dense connected layer.
Applies dense connected layer for the input. This layer implements the operation as:
\[\text{outputs} = \text{activation}(\text{inputs} * \text{kernel} + \text{bias}),\]where \(\text{activation}\) is the activation function passed as the activation argument (if passed in), \(\text{kernel}\) is a weight matrix with the same data type as the inputs created by the layer, and \(\text{bias}\) is a bias vector with the same data type as the inputs created by the layer (only if has_bias is True).
- Parameters
in_channels (int) – The number of channels in the input space.
out_channels (int) – The number of channels in the output space.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – The trainable weight_init parameter. The dtype is same as input x. The values of str refer to the function initializer. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – The trainable bias_init parameter. The dtype is same as input x. The values of str refer to the function initializer. Default: ‘zeros’.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: True.
activation (Union[str, Cell, Primitive]) – activate function applied to the output of the fully connected layer, eg. ‘ReLU’.Default: None.
- Inputs:
input (Tensor) - Tensor of shape \((*, in\_channels)\).
- Outputs:
Tensor of shape \((*, out\_channels)\).
- Raises
TypeError – If in_channels or out_channels is not an int.
TypeError – If has_bias is not a bool.
TypeError – If activation is not one of str, Cell, Primitive, None.
ValueError – If length of shape of weight_init is not equal to 2 or shape[0] of weight_init is not equal to out_channels or shape[1] of weight_init is not equal to in_channels.
ValueError – If length of shape of bias_init is not equal to 1 or shape[0] of bias_init is not equal to out_channels.
- Supported Platforms:
AscendGPUCPU
Examples
>>> input = Tensor(np.array([[180, 234, 154], [244, 48, 247]]), mindspore.float32) >>> net = nn.Dense(3, 4) >>> output = net(input) >>> print(output.shape) (2, 4)
-
class
tinyms.layers.ClipByNorm(axis=None)[source]¶ Clips tensor values to a maximum \(L_2\)-norm.
The output of this layer remains the same if the \(L_2\)-norm of the input tensor is not greater than the argument clip_norm. Otherwise the tensor will be normalized as:
\[\text{output}(X) = \frac{\text{clip_norm} * X}{L_2(X)},\]where \(L_2(X)\) is the \(L_2\)-norm of \(X\).
- Parameters
axis (Union[None, int, tuple(int)]) – Compute the L2-norm along the Specific dimension. Default: None, all dimensions to calculate.
- Inputs:
input (Tensor) - Tensor of shape N-D. The type must be float32 or float16.
clip_norm (Tensor) - A scalar Tensor of shape \(()\) or \((1)\). Or a tensor shape can be broadcast to input shape.
- Outputs:
Tensor, clipped tensor with the same shape as the input, whose type is float32.
- Raises
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.ClipByNorm() >>> input = Tensor(np.random.randint(0, 10, [4, 16]), mindspore.float32) >>> clip_norm = Tensor(np.array([100]).astype(np.float32)) >>> output = net(input, clip_norm) >>> print(output.shape) (4, 16)
-
class
tinyms.layers.Norm(axis=(), keep_dims=False)[source]¶ Computes the norm of vectors, currently including Euclidean norm, i.e., \(L_2\)-norm.
\[norm(x) = \sqrt{\sum_{i=1}^{n} (x_i^2)}\]- Parameters
- Inputs:
input (Tensor) - Tensor which is not empty.
- Outputs:
Tensor, output tensor with dimensions in ‘axis’ reduced to 1 will be returned if ‘keep_dims’ is True; otherwise a Tensor with dimensions in ‘axis’ removed is returned.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.Norm(axis=0) >>> input = Tensor(np.array([[4, 4, 9, 1], [2, 1, 3, 6]]), mindspore.float32) >>> output = net(input) >>> print(output) [4.472136 4.1231055 9.486833 6.0827627]
-
class
tinyms.layers.OneHot(axis=-1, depth=1, on_value=1.0, off_value=0.0, dtype=mindspore.float32)[source]¶ Returns a one-hot tensor.
The locations represented by indices in argument indices take value on_value, while all other locations take value off_value.
Note
If the input indices is rank \(N\), the output will have rank \(N+1\). The new axis is created at dimension axis.
If indices is a scalar, the output shape will be a vector of length depth.
If indices is a vector of length features, the output shape will be:
features * depth if axis == -1 depth * features if axis == 0
If indices is a matrix with shape [batch, features], the output shape will be:
batch * features * depth if axis == -1 batch * depth * features if axis == 1 depth * batch * features if axis == 0
- Parameters
axis (int) – Features x depth if axis is -1, depth x features if axis is 0. Default: -1.
depth (int) – A scalar defining the depth of the one hot dimension. Default: 1.
on_value (float) – A scalar defining the value to fill in output[i][j] when indices[j] = i. Default: 1.0.
off_value (float) – A scalar defining the value to fill in output[i][j] when indices[j] != i. Default: 0.0.
dtype (
mindspore.dtype) – Data type of ‘on_value’ and ‘off_value’, not the data type of indices. Default: mindspore.float32.
- Inputs:
indices (Tensor) - A tensor of indices with data type of int32 or int64 and arbitrary shape.
- Outputs:
Tensor, the one-hot tensor of data type dtype with dimension at axis expanded to depth and filled with on_value and off_value.
- Raises
TypeError – If axis or depth is not an int.
TypeError – If dtype of indices is neither int32 nor int64.
ValueError – If axis is not in range [-1, len(indices_shape)].
ValueError – If depth is less than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.OneHot(depth=4, axis=1) >>> indices = Tensor([[1, 3], [0, 2]], dtype=mindspore.int32) >>> output = net(indices) >>> print(output) [[[0. 0.] [1. 0.] [0. 0.] [0. 1.]] [[1. 0.] [0. 0.] [0. 1.] [0. 0.]]]
-
class
tinyms.layers.Pad(paddings, mode='CONSTANT')[source]¶ Pads the input tensor according to the paddings and mode.
- Parameters
paddings (tuple) – The shape of parameter paddings is (N, 2). N is the rank of input data. All elements of paddings are int type. For D th dimension of input, paddings[D, 0] indicates how many sizes to be extended ahead of the D th dimension of the input tensor, and paddings[D, 1] indicates how many sizes to be extended behind of the D th dimension of the input tensor. The padded size of each dimension D of the output is: \(paddings[D, 0] + input\_x.dim\_size(D) + paddings[D, 1]\)
mode (str) – Specifies padding mode. The optional values are “CONSTANT”, “REFLECT”, “SYMMETRIC”. Default: “CONSTANT”.
- Inputs:
input_x (Tensor) - The input tensor.
- Outputs:
Tensor, the tensor after padding.
If mode is “CONSTANT”, it fills the edge with 0, regardless of the values of the input_x. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[0,0,0,0,0,0,0], [0,0,1,2,3,0,0], [0,0,4,5,6,0,0], [0,0,7,8,9,0,0], [0,0,0,0,0,0,0]].
If mode is “REFLECT”, it uses a way of symmetrical copying through the axis of symmetry to fill in. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[6,5,4,5,6,5,4], [3,2,1,2,3,2,1], [6,5,4,5,6,5,4], [9,8,7,8,9,8,7], [6,5,4,5,6,5,4]].
If mode is “SYMMETRIC”, the filling method is similar to the “REFLECT”. It is also copied according to the symmetry axis, except that it includes the symmetry axis. If the input_x is [[1,2,3], [4,5,6], [7,8,9]] and paddings is [[1,1], [2,2]], then the Outputs is [[2,1,1,2,3,3,2], [2,1,1,2,3,3,2], [5,4,4,5,6,6,5], [8,7,7,8,9,9,8], [8,7,7,8,9,9,8]].
- Raises
TypeError – If paddings is not a tuple.
ValueError – If length of paddings is more than 4 or its shape is not (n, 2).
ValueError – If mode is not one of ‘CONSTANT’, ‘REFLECT’, ‘SYMMETRIC’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> from mindspore import Tensor >>> from mindspore.ops import operations as P >>> import mindspore.nn as nn >>> import numpy as np >>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.pad = nn.Pad(paddings=((1, 1), (2, 2)), mode="CONSTANT") ... def construct(self, x): ... return self.pad(x) >>> x = np.array([[0.3, 0.5, 0.2], [0.5, 0.7, 0.3]], dtype=np.float32) >>> pad = Net() >>> output = pad(Tensor(x)) >>> print(output) [[0. 0. 0. 0. 0. 0. 0. ] [0. 0. 0.3 0.5 0.2 0. 0. ] [0. 0. 0.5 0.7 0.3 0. 0. ] [0. 0. 0. 0. 0. 0. 0. ]]
-
class
tinyms.layers.Unfold(ksizes, strides, rates, padding='valid')[source]¶ Extracts patches from images. The input tensor must be a 4-D tensor and the data format is NCHW.
- Parameters
ksizes (Union[tuple[int], list[int]]) – The size of sliding window, must be a tuple or a list of integers, and the format is [1, ksize_row, ksize_col, 1].
strides (Union[tuple[int], list[int]]) – Distance between the centers of the two consecutive patches, must be a tuple or list of int, and the format is [1, stride_row, stride_col, 1].
rates (Union[tuple[int], list[int]]) – In each extracted patch, the gap between the corresponding dimension pixel positions, must be a tuple or a list of integers, and the format is [1, rate_row, rate_col, 1].
padding (str) –
The type of padding algorithm, is a string whose value is “same” or “valid”, not case sensitive. Default: “valid”.
same: Means that the patch can take the part beyond the original image, and this part is filled with 0.
valid: Means that the taken patch area must be completely covered in the original image.
- Inputs:
input_x (Tensor) - A 4-D tensor whose shape is [in_batch, in_depth, in_row, in_col] and data type is number.
- Outputs:
Tensor, a 4-D tensor whose data type is same as input_x, and the shape is [out_batch, out_depth, out_row, out_col] where out_batch is the same as the in_batch.
\(out\_depth = ksize\_row * ksize\_col * in\_depth\)
\(out\_row = (in\_row - (ksize\_row + (ksize\_row - 1) * (rate\_row - 1))) // stride\_row + 1\)
\(out\_col = (in\_col - (ksize\_col + (ksize\_col - 1) * (rate\_col - 1))) // stride\_col + 1\)
- Raises
TypeError – If ksizes, strides or rates is neither a tuple nor list.
ValueError – If shape of ksizes, strides or rates is not (1, x_row, x_col, 1).
ValueError – If the second and third element of ksizes, strides or rates is less than 1.
- Supported Platforms:
Ascend
Examples
>>> net = Unfold(ksizes=[1, 2, 2, 1], strides=[1, 2, 2, 1], rates=[1, 2, 2, 1]) >>> image = Tensor(np.ones([2, 3, 6, 6]), dtype=mstype.float16) >>> output = net(image) >>> print(output.shape) (2, 12, 2, 2)
-
class
tinyms.layers.Tril[source]¶ Returns a tensor with elements above the kth diagonal zeroed.
- Inputs:
x (Tensor) - The input tensor.
k (Int) - The index of diagonal. Default: 0
- Outputs:
Tensor, has the same type as input x.
- Raises
TypeError – If k is not an int.
ValueError – If length of shape of x is less than 1.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([[1, 2], [3, 4]])) >>> tril = nn.Tril() >>> result = tril(x) >>> print(result) [[1 0] [3 4]]
-
class
tinyms.layers.Triu[source]¶ Returns a tensor with elements below the kth diagonal zeroed.
- Inputs:
x (Tensor) - The input tensor.
k (Int) - The index of diagonal. Default: 0
- Outputs:
Tensor, has the same type as input x.
- Raises
TypeError – If k is not an int.
ValueError – If length of shape of x is less than 1.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([[1, 2], [3, 4]])) >>> triu = nn.Triu() >>> result = triu(x) >>> print(result) [[1 2] [0 4]]
-
class
tinyms.layers.ResizeBilinear[source]¶ Samples the input tensor to the given size or scale_factor by using bilinear interpolate.
- Inputs:
x (Tensor) - Tensor to be resized. Input tensor must be a 4-D tensor with shape: math:(batch, channels, height, width), with data type of float16 or float32.
size (Union[tuple[int], list[int]]): A tuple or list of 2 int elements ‘(new_height, new_width)’, the new size of the tensor. One and only one of size and scale_factor can be set to None. Default: None.
scale_factor (int): The scale factor of new size of the tensor. The value should be positive integer. One and only one of size and scale_factor can be set to None. Default: None.
align_corners (bool): If true, rescale input by ‘(new_height - 1) / (height - 1)’, which exactly aligns the 4 corners of images and resized images. If false, rescale by ‘new_height / height’. Default: False.
- Outputs:
Resized tensor. If size is set, the result is 4-D tensor with shape:math:(batch, channels, new_height, new_width) in float32. If scale is set, the result is 4-D tensor with shape:math:(batch, channels, scale_factor * height, scale_factor * width) in float32
- Raises
TypeError – If size is not one of tuple, list, None.
TypeError – If scale_factor is neither int nor None.
TypeError – If align_corners is not a bool.
TypeError – If dtype of x is neither float16 nor float32.
ValueError – If size and scale_factor are both None or not None.
ValueError – If length of shape of x is not equal to 4.
ValueError – If scale_factor is an int which is less than 1.
ValueError – If size is a list or tuple whose length is not equal to 2.
- Supported Platforms:
AscendCPU
Examples
>>> tensor = Tensor([[[[1, 2, 3, 4], [5, 6, 7, 8]]]], mindspore.float32) >>> resize_bilinear = nn.ResizeBilinear() >>> result = resize_bilinear(tensor, size=(5,5)) >>> print(result.shape) (1, 1, 5, 5)
-
class
tinyms.layers.MatrixDiag[source]¶ Returns a batched diagonal tensor with a given batched diagonal values.
Assume x has \(k\) dimensions \([I, J, K, ..., N]\), then the output is a tensor of rank \(k+1\) with dimensions \([I, J, K, ..., N, N]\) where: \(output[i, j, k, ..., m, n] = 1\{m=n\} * x[i, j, k, ..., n]\)
- Inputs:
x (Tensor) - The diagonal values. It can be one of the following data types: float32, float16, int32, int8, and uint8.
- Outputs:
Tensor, has the same type as input x. The shape must be x.shape + (x.shape[-1], ).
- Raises
TypeError – If dtype of x is not one of float32, float16, int32, int8 or uint8.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor(np.array([1, -1]), mindspore.float32) >>> matrix_diag = nn.MatrixDiag() >>> output = matrix_diag(x) >>> print(output) [[ 1. 0.] [ 0. -1.]]
-
class
tinyms.layers.MatrixDiagPart[source]¶ Returns the batched diagonal part of a batched tensor.
Assume x has \(k\) dimensions \([I, J, K, ..., M, N]\), then the output is a tensor of rank \(k-1\) with dimensions \([I, J, K, ..., min(M, N)]\) where: \(output[i, j, k, ..., n] = x[i, j, k, ..., n, n]\)
- Inputs:
x (Tensor) - The batched tensor. It can be one of the following data types: float32, float16, int32, int8, and uint8.
- Outputs:
Tensor, has the same type as input x. The shape must be x.shape[:-2] + [min(x.shape[-2:])].
- Raises
TypeError – If dtype of x is not one of float32, float16, int32, int8 or uint8.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor([[[-1, 0], [0, 1]], [[-1, 0], [0, 1]], [[-1, 0], [0, 1]]], mindspore.float32) >>> matrix_diag_part = nn.MatrixDiagPart() >>> output = matrix_diag_part(x) >>> print(output) [[-1. 1.] [-1. 1.] [-1. 1.]]
-
class
tinyms.layers.MatrixSetDiag[source]¶ Modifies the batched diagonal part of a batched tensor.
Assume x has \(k+1\) dimensions \([I, J, K, ..., M, N]\) and diagonal has \(k\) dimensions \([I, J, K, ..., min(M, N)]\). Then the output is a tensor of rank \(k+1\) with dimensions \([I, J, K, ..., M, N]\) where:
\[output[i, j, k, ..., m, n] = diagnoal[i, j, k, ..., n]\ for\ m == n\]\[output[i, j, k, ..., m, n] = x[i, j, k, ..., m, n]\ for\ m != n\]- Inputs:
x (Tensor) - The batched tensor. Rank k+1, where k >= 1. It can be one of the following data types: float32, float16, int32, int8, and uint8.
diagonal (Tensor) - The diagonal values. Must have the same type as input x. Rank k, where k >= 1.
- Outputs:
Tensor, has the same type and shape as input x.
- Raises
TypeError – If dtype of x or diagonal is not one of float32, float16, int32, int8 or uint8.
ValueError – If length of shape of x is less than 2.
ValueError – If x_shape[-2] < x_shape[-1] and x_shape[:-1] != diagonal_shape.
ValueError – If x_shape[-2] >= x_shape[-1] and x_shape[:-2] + x_shape[-1:] != diagonal_shape.
- Supported Platforms:
Ascend
Examples
>>> x = Tensor([[[-1, 0], [0, 1]], [[-1, 0], [0, 1]], [[-1, 0], [0, 1]]], mindspore.float32) >>> diagonal = Tensor([[-1., 2.], [-1., 1.], [-1., 1.]], mindspore.float32) >>> matrix_set_diag = nn.MatrixSetDiag() >>> output = matrix_set_diag(x, diagonal) >>> print(output) [[[-1. 0.] [ 0. 2.]] [[-1. 0.] [ 0. 1.]] [[-1. 0.] [ 0. 1.]]]
-
class
tinyms.layers.L1Regularizer(scale)[source]¶ Applies l1 regularization to weights.
l1 regularization makes weights sparsity
\[\text{loss}=\lambda * \text{reduce_sum}(\text{abs}(\omega))\]Note
scale(regularization factor) should be a number which greater than 0
- Inputs:
weights (Tensor) - The input tensor
- Outputs:
Tensor, which dtype is higher precision data type between mindspore.float32 and weights dtype, and Tensor shape is ()
- Raises
TypeError – If scale is neither an int nor float.
ValueError – If scale is not greater than 0.
ValueError – If scale is math.inf or math.nan.
- Supported Platforms:
AscendGPUCPU
Examples
>>> scale = 0.5 >>> net = nn.L1Regularizer(scale) >>> weights = Tensor(np.array([[1.0, -2.0], [-3.0, 4.0]]).astype(np.float32)) >>> output = net(weights) >>> print(output.asnumpy()) 5.0
-
class
tinyms.layers.Embedding(vocab_size, embedding_size, use_one_hot=False, embedding_table='normal', dtype=mindspore.float32, padding_idx=None)[source]¶ A simple lookup table that stores embeddings of a fixed dictionary and size.
This module is often used to store word embeddings and retrieve them using indices. The input to the module is a list of indices, and the output is the corresponding word embeddings.
Note
When ‘use_one_hot’ is set to True, the type of the input must be mindspore.int32.
- Parameters
vocab_size (int) – Size of the dictionary of embeddings.
embedding_size (int) – The size of each embedding vector.
use_one_hot (bool) – Specifies whether to apply one_hot encoding form. Default: False.
embedding_table (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the embedding_table. Refer to class initializer for the values of string when a string is specified. Default: ‘normal’.
dtype (
mindspore.dtype) – Data type of input. Default: mindspore.float32.padding_idx (int, None) – When the padding_idx encounters index, the output embedding vector of this index will be initialized to zero. Default: None. The feature is inactivated.
- Inputs:
input (Tensor) - Tensor of shape \((\text{batch_size}, \text{input_length})\). The elements of the Tensor must be integer and not larger than vocab_size. Otherwise the corresponding embedding vector will be zero.
- Outputs:
Tensor of shape \((\text{batch_size}, \text{input_length}, \text{embedding_size})\).
- Raises
TypeError – If vocab_size or embedding_size is not an int.
TypeError – If use_one_hot is not a bool.
ValueError – If padding_idx is an int which not in range [0, vocab_size].
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Embedding(20000, 768, True) >>> input_data = Tensor(np.ones([8, 128]), mindspore.int32) >>> >>> # Maps the input word IDs to word embedding. >>> output = net(input_data) >>> result = output.shape >>> print(result) (8, 128, 768)
-
class
tinyms.layers.EmbeddingLookup(vocab_size, embedding_size, param_init='normal', target='CPU', slice_mode='batch_slice', manual_shapes=None, max_norm=None, sparse=True, vocab_cache_size=0)[source]¶ Returns a slice of the input tensor based on the specified indices.
Note
When ‘target’ is set to ‘CPU’, this module will use P.EmbeddingLookup().add_prim_attr(‘primitive_target’, ‘CPU’) which specified ‘offset = 0’ to lookup table. When ‘target’ is set to ‘DEVICE’, this module will use P.Gather() which specified ‘axis = 0’ to lookup table. In field slice mode, the manual_shapes must be given. It is a tuple ,where the element is vocab[i], vocab[i] is the row numbers for i-th part.
- Parameters
vocab_size (int) – Size of the dictionary of embeddings.
embedding_size (int) – The size of each embedding vector.
param_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the embedding_table. Refer to class initializer for the values of string when a string is specified. Default: ‘normal’.
target (str) – Specifies the target where the op is executed. The value must in [‘DEVICE’, ‘CPU’]. Default: ‘CPU’.
slice_mode (str) – The slicing way in semi_auto_parallel/auto_parallel. The value must get through nn.EmbeddingLookup. Default: nn.EmbeddingLookup.BATCH_SLICE.
manual_shapes (tuple) – The accompaniment array in field slice mode.
max_norm (Union[float, None]) – A maximum clipping value. The data type must be float16, float32 or None. Default: None
sparse (bool) – Using sparse mode. When ‘target’ is set to ‘CPU’, ‘sparse’ has to be true. Default: True.
vocab_cache_size (int) – Cache size of the dictionary of embeddings. Default: 0. It is valid only in ‘DEVICE’ target. And the moment parameter of corresponding optimizer will also be set to the cache size. In addition, it should be noted that it will cost the ‘DEVICE’ memory, so suggests setting a reasonable value to avoid insufficient memory.
- Inputs:
input_indices (Tensor) - The shape of tensor is \((y_1, y_2, ..., y_S)\). Specifies the indices of elements of the original Tensor. Values can be out of range of embedding_table, and the exceeding part will be filled with 0 in the output. Values does not support negative and the result is undefined if values are negative. Input_indices must only be a 2d tensor in this interface when run in semi auto parallel/auto parallel mode.
- Outputs:
Tensor, the shape of tensor is \((z_1, z_2, ..., z_N)\).
- Raises
TypeError – If vocab_size or embedding_size or vocab_cache_size is not an int.
TypeError – If sparse is not a bool or manual_shapes is not a tuple.
ValueError – If vocab_size or embedding_size is less than 1.
ValueError – If vocab_cache_size is less than 0.
ValueError – If target is neither ‘CPU’ nor ‘DEVICE’.
ValueError – If slice_mode is not one of ‘batch_slice’ or ‘field_slice’ or ‘table_row_slice’ or ‘table_column_slice’.
ValueError – If sparse is False and target is ‘CPU’.
ValueError – If slice_mode is ‘field_slice’ and manual_shapes is None.
- Supported Platforms:
AscendCPU
Examples
>>> input_indices = Tensor(np.array([[1, 0], [3, 2]]), mindspore.int32) >>> result = nn.EmbeddingLookup(4,2)(input_indices) >>> print(result.shape) (2, 2, 2)
-
class
tinyms.layers.MultiFieldEmbeddingLookup(vocab_size, embedding_size, field_size, param_init='normal', target='CPU', slice_mode='batch_slice', feature_num_list=None, max_norm=None, sparse=True, operator='SUM')[source]¶ Returns a slice of input tensor based on the specified indices and the field ids. This operation supports looking up embeddings using multi hot and one hot fields simultaneously.
Note
When ‘target’ is set to ‘CPU’, this module will use P.EmbeddingLookup().add_prim_attr(‘primitive_target’, ‘CPU’) which specified ‘offset = 0’ to lookup table. When ‘target’ is set to ‘DEVICE’, this module will use P.Gather() which specified ‘axis = 0’ to lookup table. The vectors with the same field_ids will be combined by the ‘operator’, such as ‘SUM’, ‘MAX’ and ‘MEAN’. Ensure the input_values of the padded id is zero, so that they can be ignored. The final output will be zeros if the sum of absolute weight of the field is zero. This class only supports [‘table_row_slice’, ‘batch_slice’ and ‘table_column_slice’]. For the operation ‘MAX’ on device Ascend, there is a constrain where batch_size * (seq_length + field_size) < 3500.
- Parameters
vocab_size (int) – The size of the dictionary of embeddings.
embedding_size (int) – The size of each embedding vector.
field_size (int) – The field size of the final outputs.
param_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the embedding_table. Refer to class initializer for the values of string when a string is specified. Default: ‘normal’.
target (str) – Specifies the target where the op is executed. The value must in [‘DEVICE’, ‘CPU’]. Default: ‘CPU’.
slice_mode (str) – The slicing way in semi_auto_parallel/auto_parallel. The value must get through nn.EmbeddingLookup. Default: nn.EmbeddingLookup.BATCH_SLICE.
feature_num_list (tuple) – The accompaniment array in field slice mode. This is unused currently.
max_norm (Union[float, None]) – A maximum clipping value. The data type must be float16, float32 or None. Default: None
sparse (bool) – Using sparse mode. When ‘target’ is set to ‘CPU’, ‘sparse’ has to be true. Default: True.
operator (str) – The pooling method for the features in one field. Support ‘SUM, ‘MEAN’ and ‘MAX’
- Inputs:
input_indices (Tensor) - The shape of tensor is \((batch\_size, seq\_length)\). Specifies the indices of elements of the original Tensor. Input_indices must be a 2d tensor in this interface. Type is Int32, Int64.
input_values (Tensor) - The shape of tensor is \((batch\_size, seq\_length)\). Specifies the weights of elements of the input_indices. The lookout vector will multiply with the input_values. Type is Float32.
field_ids (Tensor) - The shape of tensor is \((batch\_size, seq\_length)\). Specifies the field id of elements of the input_indices. Type is Int32.
- Outputs:
Tensor, the shape of tensor is \((batch\_size, field\_size, embedding\_size)\). Type is Float32.
- Raises
TypeError – If vocab_size or embedding_size or field_size is not an int.
TypeError – If sparse is not a bool or feature_num_list is not a tuple.
ValueError – If vocab_size or embedding_size or field_size is less than 1.
ValueError – If target is neither ‘CPU’ nor ‘DEVICE’.
ValueError – If slice_mode is not one of ‘batch_slice’, ‘field_slice’, ‘table_row_slice’, ‘table_column_slice’.
ValueError – If sparse is False and target is ‘CPU’.
ValueError – If slice_mode is ‘field_slice’ and feature_num_list is None.
ValueError – If operator is not one of ‘SUM’, ‘MAX’, ‘MEAN’.
- Supported Platforms:
AscendGPU
Examples
>>> input_indices = Tensor([[2, 4, 6, 0, 0], [1, 3, 5, 0, 0]], mindspore.int32) >>> input_values = Tensor([[1, 1, 1, 0, 0], [1, 1, 1, 0, 0]], mindspore.float32) >>> field_ids = Tensor([[0, 1, 1, 0, 0], [0, 0, 1, 0, 0]], mindspore.int32) >>> net = nn.MultiFieldEmbeddingLookup(10, 2, field_size=2, operator='SUM', target='DEVICE') >>> out = net(input_indices, input_values, field_ids) >>> print(out.shape) (2, 2, 2)
-
class
tinyms.layers.AvgPool2d(kernel_size=1, stride=1, pad_mode='valid', data_format='NCHW')[source]¶ 2D average pooling for temporal data.
Applies a 2D average pooling over an input Tensor which can be regarded as a composition of 2D input planes.
Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), AvgPool2d outputs regional average in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \frac{1}{h_{ker} * w_{ker}} \sum_{m=0}^{h_{ker}-1} \sum_{n=0}^{w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]Note
pad_mode for training only supports “same” and “valid”.
- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the average value. The data type of kernel_size must be int and the value represents the height and width, or a tuple of two int numbers that represent height and width respectively. Default: 1.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If kernel_size or strides is neither int nor tuple.
ValueError – If pad_mode is neither ‘valid’ nor ‘same’ with not case sensitive.
ValueError – If data_format is neither ‘NCHW’ nor ‘NHWC’.
ValueError – If kernel_size or strides is less than 1.
ValueError – If length of shape of input is not equal to 4.
- Supported Platforms:
AscendGPUCPU
Examples
>>> pool = nn.AvgPool2d(kernel_size=3, stride=1) >>> x = Tensor(np.random.randint(0, 10, [1, 2, 4, 4]), mindspore.float32) >>> output = pool(x) >>> print(output.shape) (1, 2, 2, 2)
-
class
tinyms.layers.MaxPool2d(kernel_size=1, stride=1, pad_mode='valid', data_format='NCHW')[source]¶ 2D max pooling operation for temporal data.
Applies a 2D max pooling over an input Tensor which can be regarded as a composition of 2D planes.
Typically the input is of shape \((N_{in}, C_{in}, H_{in}, W_{in})\), MaxPool2d outputs regional maximum in the \((H_{in}, W_{in})\)-dimension. Given kernel size \(ks = (h_{ker}, w_{ker})\) and stride \(s = (s_0, s_1)\), the operation is as follows.
\[\text{output}(N_i, C_j, h, w) = \max_{m=0, \ldots, h_{ker}-1} \max_{n=0, \ldots, w_{ker}-1} \text{input}(N_i, C_j, s_0 \times h + m, s_1 \times w + n)\]Note
pad_mode for training only supports “same” and “valid”.
- Parameters
kernel_size (Union[int, tuple[int]]) – The size of kernel used to take the max value, is an int number that represents height and width are both kernel_size, or a tuple of two int numbers that represent height and width respectively. Default: 1.
stride (Union[int, tuple[int]]) – The distance of kernel moving, an int number that represents the height and width of movement are both strides, or a tuple of two int numbers that represent height and width of movement respectively. Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
data_format (str) – The optional value for data format, is ‘NHWC’ or ‘NCHW’. Default: ‘NCHW’.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If kernel_size or strides is neither int nor tuple.
ValueError – If pad_mode is neither ‘valid’ nor ‘same’ with not case sensitive.
ValueError – If data_format is neither ‘NCHW’ nor ‘NHWC’.
ValueError – If kernel_size or strides is less than 1.
ValueError – If length of shape of input is not equal to 4.
- Supported Platforms:
AscendGPUCPU
Examples
>>> pool = nn.MaxPool2d(kernel_size=3, stride=1) >>> x = Tensor(np.random.randint(0, 10, [1, 2, 4, 4]), mindspore.float32) >>> output = pool(x) >>> print(output.shape) (1, 2, 2, 2)
-
class
tinyms.layers.AvgPool1d(kernel_size=1, stride=1, pad_mode='valid')[source]¶ 1D average pooling for temporal data.
Applies a 1D average pooling over an input Tensor which can be regarded as a composition of 1D input planes.
Typically the input is of shape \((N_{in}, C_{in}, L_{in})\), AvgPool1d outputs regional average in the \((L_{in})\)-dimension. Given kernel size \(ks = l_{ker}\) and stride \(s = s_0\), the operation is as follows.
\[\text{output}(N_i, C_j, l) = \frac{1}{l_{ker}} \sum_{n=0}^{l_{ker}-1} \text{input}(N_i, C_j, s_0 \times l + n)\]Note
pad_mode for training only supports “same” and “valid”.
- Parameters
kernel_size (int) – The size of kernel window used to take the average value, Default: 1.
stride (int) – The distance of kernel moving, an int number that represents the width of movement is strides, Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The height and width of the output will be the same as the input. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, L_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, L_{out})\).
- Raises
TypeError – If kernel_size or stride is not an int.
ValueError – If pad_mode is neither ‘same’ nor ‘valid’ with not case sensitive.
ValueError – If kernel_size or strides is less than 1.
ValueError – If length of shape of input is not equal to 3.
- Supported Platforms:
AscendGPUCPU
Examples
>>> pool = nn.AvgPool1d(kernel_size=6, stride=1) >>> x = Tensor(np.random.randint(0, 10, [1, 3, 6]), mindspore.float32) >>> output = pool(x) >>> result = output.shape >>> print(result) (1, 3, 1)
-
class
tinyms.layers.MaxPool1d(kernel_size=1, stride=1, pad_mode='valid')[source]¶ 1D max pooling operation for temporal data.
Applies a 1D max pooling over an input Tensor which can be regarded as a composition of 1D planes.
Typically the input is of shape \((N_{in}, C_{in}, L_{in})\), MaxPool1d outputs regional maximum in the \((L_{in})\)-dimension. Given kernel size \(ks = (l_{ker})\) and stride \(s = (s_0)\), the operation is as follows.
\[\text{output}(N_i, C_j, l) = \max_{n=0, \ldots, l_{ker}-1} \text{input}(N_i, C_j, s_0 \times l + n)\]Note
pad_mode for training only supports “same” and “valid”.
- Parameters
kernel_size (int) – The size of kernel used to take the max value, Default: 1.
stride (int) – The distance of kernel moving, an int number that represents the width of movement is stride, Default: 1.
pad_mode (str) –
The optional value for pad mode, is “same” or “valid”, not case sensitive. Default: “valid”.
same: Adopts the way of completion. The total number of padding will be calculated in horizontal and vertical directions and evenly distributed to top and bottom, left and right if possible. Otherwise, the last extra padding will be done from the bottom and the right side.
valid: Adopts the way of discarding. The possible largest height and width of output will be returned without padding. Extra pixels will be discarded.
- Inputs:
input (Tensor) - Tensor of shape \((N, C, L_{in})\).
- Outputs:
Tensor of shape \((N, C, L_{out}))\).
- Raises
TypeError – If kernel_size or strides is not an int.
ValueError – If pad_mode is neither ‘valid’ nor ‘same’ with not case sensitive.
ValueError – If data_format is neither ‘NCHW’ nor ‘NHWC’.
ValueError – If kernel_size or strides is less than 1.
ValueError – If length of shape of input is not equal to 4.
- Supported Platforms:
AscendGPUCPU
Examples
>>> max_pool = nn.MaxPool1d(kernel_size=3, stride=1) >>> x = Tensor(np.random.randint(0, 10, [1, 2, 4]), mindspore.float32) >>> output = max_pool(x) >>> result = output.shape >>> print(result) (1, 2, 2)
-
class
tinyms.layers.ReduceLogSumExp(axis, keep_dims=False)[source]¶ Reduces a dimension of a tensor by calculating exponential for all elements in the dimension, then calculate logarithm of the sum.
The dtype of the tensor to be reduced is number.
\[ReduceLogSumExp(x) = \log(\sum(e^x))\]- Parameters
- Inputs:
x (Tensor) - The input tensor. With float16 or float32 data type.
- Outputs:
Tensor, has the same dtype as the x.
If axis is (), and keep_dims is False, the output is a 0-D tensor representing the sum of all elements in the input tensor.
If axis is int, set as 2, and keep_dims is False, the shape of output is \((x_1, x_3, ..., x_R)\).
If axis is tuple(int), set as (2, 3), and keep_dims is False, the shape of output is \((x_1, x_4, ..., x_R)\).
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.random.randn(3, 4, 5, 6).astype(np.float32)) >>> op = nn.ReduceLogSumExp(1, keep_dims=True) >>> output = op(input_x) >>> print(output.shape) (3, 1, 5, 6)
-
class
tinyms.layers.Range(start, limit=None, delta=1)[source]¶ Creates a sequence of numbers in range [start, limit) with step size delta.
The size of output is \(\left \lfloor \frac{limit-start}{delta} \right \rfloor + 1\) and delta is the gap between two values in the tensor.
\[out_{i+1} = out_{i} +delta\]- Parameters
start (Union[int, float]) – If limit is None, the value acts as limit in the range and first entry defaults to 0. Otherwise, it acts as first entry in the range.
limit (Union[int, float]) – Acts as upper limit of sequence. If None, defaults to the value of start while set the first entry of the range to 0. It can not be equal to start.
delta (Union[int, float]) – Increment of the range. It can not be equal to zero. Default: 1.
- Outputs:
Tensor, the dtype is int if the dtype of start, limit and delta all are int. Otherwise, dtype is float.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.Range(1, 8, 2) >>> output = net() >>> print(output) [1 3 5 7]
-
class
tinyms.layers.LGamma[source]¶ Calculates LGamma using Lanczos’ approximation referring to “A Precision Approximation of the Gamma Function”. The algorithm is:
\[\begin{split}\begin{array}{ll} \\ lgamma(z + 1) = \frac{(\log(2) + \log(pi))}{2} + (z + 1/2) * log(t(z)) - t(z) + A(z) \\ t(z) = z + kLanczosGamma + 1/2 \\ A(z) = kBaseLanczosCoeff + \sum_{k=1}^n \frac{kLanczosCoefficients[i]}{z + k} \end{array}\end{split}\]However, if the input is less than 0.5 use Euler’s reflection formula:
\[lgamma(x) = \log(pi) - lgamma(1-x) - \log(abs(sin(pi * x)))\]And please note that
\[lgamma(+/-inf) = +inf\]Thus, the behaviour of LGamma follows: when x > 0.5, return log(Gamma(x)) when x < 0.5 and is not an integer, return the real part of Log(Gamma(x)) where Log is the complex logarithm when x is an integer less or equal to 0, return +inf when x = +/- inf, return +inf
- Inputs:
x (Tensor) - The input tensor. Only float16, float32 are supported.
- Outputs:
Tensor, has the same shape and dtype as the x.
- Raises
TypeError – If dtype of x is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([2, 3, 4]).astype(np.float32)) >>> op = nn.LGamma() >>> output = op(input_x) >>> print(output) [3.5762787e-07 6.9314754e-01 1.7917603e+00]
-
class
tinyms.layers.DiGamma[source]¶ Calculates Digamma using Lanczos’ approximation referring to “A Precision Approximation of the Gamma Function”. The algorithm is:
\[\begin{split}\begin{array}{ll} \\ digamma(z + 1) = log(t(z)) + A'(z) / A(z) - kLanczosGamma / t(z) \\ t(z) = z + kLanczosGamma + 1/2 \\ A(z) = kBaseLanczosCoeff + \sum_{k=1}^n \frac{kLanczosCoefficients[i]}{z + k} \\ A'(z) = \sum_{k=1}^n \frac{kLanczosCoefficients[i]}{{z + k}^2} \end{array}\end{split}\]However, if the input is less than 0.5 use Euler’s reflection formula:
\[digamma(x) = digamma(1 - x) - pi * cot(pi * x)\]- Inputs:
x (Tensor[Number]) - The input tensor. Only float16, float32 are supported.
- Outputs:
Tensor, has the same shape and dtype as the x.
- Raises
TypeError – If dtype of x is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([2, 3, 4]).astype(np.float32)) >>> op = nn.DiGamma() >>> output = op(input_x) >>> print(output) [0.42278463 0.92278427 1.2561178]
-
class
tinyms.layers.IGamma[source]¶ Calculates lower regularized incomplete Gamma function. The lower regularized incomplete Gamma function is defined as:
\[P(a, x) = gamma(a, x) / Gamma(a) = 1 - Q(a, x)\]where
\[gamma(a, x) = \int_0^x t^{a-1} \exp^{-t} dt\]is the lower incomplete Gamma function.
Above \(Q(a, x)\) is the upper regularized complete Gamma function.
- Inputs:
a (Tensor) - The input tensor. With float32 data type. a should have the same dtype with x.
x (Tensor) - The input tensor. With float32 data type. x should have the same dtype with a.
- Outputs:
Tensor, has the same dtype as a and x.
- Raises
TypeError – If dtype of input x and a is not float16 nor float32, or if x has different dtype with a.
- Supported Platforms:
AscendGPU
Examples
>>> input_a = Tensor(np.array([2.0, 4.0, 6.0, 8.0]).astype(np.float32)) >>> input_x = Tensor(np.array([2.0, 3.0, 4.0, 5.0]).astype(np.float32)) >>> igamma = nn.IGamma() >>> output = igamma(input_a, input_x) >>> print (output) [0.593994 0.35276785 0.21486944 0.13337152]
-
class
tinyms.layers.LBeta[source]¶ This is semantically equal to
\[P(x, y) = lgamma(x) + lgamma(y) - lgamma(x + y).\]The method is more accurate for arguments above 8. The reason for accuracy loss in the naive computation is catastrophic cancellation between the lgammas. This method avoids the numeric cancellation by explicitly decomposing lgamma into the Stirling approximation and an explicit log_gamma_correction, and cancelling the large terms from the Striling analytically.
- Inputs:
x (Tensor) - The input tensor. With float16 or float32 data type. x should have the same dtype with y.
y (Tensor) - The input tensor. With float16 or float32 data type. y should have the same dtype with x.
- Outputs:
Tensor, has the same dtype as x and y.
- Raises
TypeError – If dtype of x or y is neither float16 nor float32, or if x has different dtype with y.
- Supported Platforms:
AscendGPU
Examples
>>> input_x = Tensor(np.array([2.0, 4.0, 6.0, 8.0]).astype(np.float32)) >>> input_y = Tensor(np.array([2.0, 3.0, 14.0, 15.0]).astype(np.float32)) >>> lbeta = nn.LBeta() >>> output = lbeta(input_y, input_x) >>> print(output) [-1.7917596 -4.094345 -12.000229 -14.754799]
-
class
tinyms.layers.MatMul(**kwargs)[source]¶ Multiplies matrix x1 by matrix x2.
nn.MatMul will be deprecated in future versions. Please use ops.matmul instead.
If both x1 and x2 are 1-dimensional, the dot product is returned.
If the dimensions of x1 and x2 are all not greater than 2, the matrix-matrix product will be returned. Note if one of ‘x1’ and ‘x2’ is 1-dimensional, the argument will first be expanded to 2 dimension. After the matrix multiply, the expanded dimension will be removed.
If at least one of x1 and x2 is N-dimensional (N>2), the none-matrix dimensions(batch) of inputs will be broadcasted and must be broadcastable. Note if one of ‘x1’ and ‘x2’ is 1-dimensional, the argument will first be expanded to 2 dimension and then the none-matrix dimensions will be broadcasted. After the matrix multiply, the expanded dimension will be removed. For example, if x1 is a \((j \times 1 \times n \times m)\) tensor and x2 is a \((k \times m \times p)\) tensor, the output will be a \((j \times k \times n \times p)\) tensor.
- Parameters
- Inputs:
input_x1 (Tensor) - The first tensor to be multiplied.
input_x2 (Tensor) - The second tensor to be multiplied.
- Outputs:
Tensor, the shape of the output tensor depends on the dimension of input tensors.
- Raises
TypeError – If transpose_x1 or transpose_x2 is not a bool.
ValueError – If the column of matrix dimensions of input_x1 is not equal to the row of matrix dimensions of input_x2.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.MatMul() >>> input_x1 = Tensor(np.ones(shape=[3, 2, 3]), mindspore.float32) >>> input_x2 = Tensor(np.ones(shape=[3, 4]), mindspore.float32) >>> output = net(input_x1, input_x2) >>> print(output.shape) (3, 2, 4)
-
class
tinyms.layers.Moments(axis=None, keep_dims=None)[source]¶ Calculates the mean and variance of x.
- Parameters
- Inputs:
input_x (Tensor) - The tensor to be calculated. Only float16 and float32 are supported.
- Outputs:
mean (Tensor) - The mean of input x, with the same date type as input x.
variance (Tensor) - The variance of input x, with the same date type as input x.
- Raises
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.Moments(axis=3, keep_dims=True) >>> input_x = Tensor(np.array([[[[1, 2, 3, 4], [3, 4, 5, 6]]]]), mindspore.float32) >>> output = net(input_x) >>> print(output) (Tensor(shape=[1, 1, 2, 1], dtype=Float32, value= [[[[ 2.50000000e+00], [ 4.50000000e+00]]]]), Tensor(shape=[1, 1, 2, 1], dtype=Float32, value= [[[[ 1.25000000e+00], [ 1.25000000e+00]]]]))
-
class
tinyms.layers.MatInverse[source]¶ Calculates the inverse of Positive-Definite Hermitian matrix using Cholesky decomposition.
- Inputs:
a (Tensor[Number]) - The input tensor. It must be a positive-definite matrix. With float16 or float32 data type.
- Outputs:
Tensor, has the same dtype as the a.
- Raises
TypeError – If dtype of a is neither float16 nor float32.
- Supported Platforms:
GPU
Examples
>>> input_a = Tensor(np.array([[4, 12, -16], [12, 37, -43], [-16, -43, 98]]).astype(np.float32)) >>> op = nn.MatInverse() >>> output = op(input_a) >>> print(output) [[49.36112 -13.555558 2.1111116] [-13.555558 3.7777784 -0.5555557] [2.1111116 -0.5555557 0.11111111]]
-
class
tinyms.layers.MatDet[source]¶ Calculates the determinant of Positive-Definite Hermitian matrix using Cholesky decomposition.
- Inputs:
a (Tensor[Number]) - The input tensor. It must be a positive-definite matrix. With float16 or float32 data type.
- Outputs:
Tensor, has the same dtype as the a.
- Raises
TypeError – If dtype of a is neither float16 nor float32.
- Supported Platforms:
GPU
Examples
>>> input_a = Tensor(np.array([[4, 12, -16], [12, 37, -43], [-16, -43, 98]]).astype(np.float32)) >>> op = nn.MatDet() >>> output = op(input_a) >>> print(output) 35.999996
-
class
tinyms.layers.Conv2dBnAct(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros', has_bn=False, momentum=0.997, eps=1e-05, activation=None, alpha=0.2, after_fake=True)[source]¶ A combination of convolution, Batchnorm, and activation layer.
This part is a more detailed overview of Conv2d operation.
- Parameters
in_channels (int) – The number of input channel \(C_{in}\).
out_channels (int) – The number of output channel \(C_{out}\).
kernel_size (Union[int, tuple]) – The data type is int or a tuple of 2 integers. Specifies the height and width of the 2D convolution window. Single int means the value is for both height and width of the kernel. A tuple of 2 ints means the first value is for the height and the other is for the width of the kernel.
stride (int) – Specifies stride for all spatial dimensions with the same value. The value of stride must be greater than or equal to 1 and lower than any one of the height and width of the input. Default: 1.
pad_mode (str) – Specifies padding mode. The optional values are “same”, “valid”, “pad”. Default: “same”.
padding (int) – Implicit paddings on both sides of the input. Default: 0.
dilation (int) – Specifies the dilation rate to use for dilated convolution. If set to be \(k > 1\), there will be \(k - 1\) pixels skipped for each sampling location. Its value must be greater than or equal to 1 and lower than any one of the height and width of the input. Default: 1.
group (int) – Splits filter into groups, in_ channels and out_channels must be divisible by the number of groups. Default: 1.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: False.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the convolution kernel. It can be a Tensor, a string, an Initializer or a number. When a string is specified, values from ‘TruncatedNormal’, ‘Normal’, ‘Uniform’, ‘HeUniform’ and ‘XavierUniform’ distributions as well as constant ‘One’ and ‘Zero’ distributions are possible. Alias ‘xavier_uniform’, ‘he_uniform’, ‘ones’ and ‘zeros’ are acceptable. Uppercase and lowercase are both acceptable. Refer to the values of Initializer for more details. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – Initializer for the bias vector. Possible Initializer and string are the same as ‘weight_init’. Refer to the values of Initializer for more details. Default: ‘zeros’.
has_bn (bool) – Specifies to used batchnorm or not. Default: False.
momentum (float) – Momentum for moving average for batchnorm, must be [0, 1]. Default:0.9
eps (float) – Term added to the denominator to improve numerical stability for batchnorm, should be greater than 0. Default: 1e-5.
activation (Union[str, Cell, Primitive]) – Specifies activation type. The optional values are as following: ‘softmax’, ‘logsoftmax’, ‘relu’, ‘relu6’, ‘tanh’, ‘gelu’, ‘sigmoid’, ‘prelu’, ‘leakyrelu’, ‘hswish’, ‘hsigmoid’. Default: None.
alpha (float) – Slope of the activation function at x < 0 for LeakyReLU. Default: 0.2.
after_fake (bool) – Determine whether there must be a fake quantization operation after Cond2dBnAct.
- Inputs:
input (Tensor) - Tensor of shape \((N, C_{in}, H_{in}, W_{in})\).
- Outputs:
Tensor of shape \((N, C_{out}, H_{out}, W_{out})\).
- Raises
TypeError – If in_channels, out_channels, stride, padding or dilation is not an int.
TypeError – If has_bias is not a bool.
ValueError – If in_channels or out_channels stride, padding or dilation is less than 1.
ValueError – If pad_mode is not one of ‘same’, ‘valid’, ‘pad’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = nn.Conv2dBnAct(120, 240, 4, has_bn=True, activation='relu') >>> input = Tensor(np.ones([1, 120, 1024, 640]), mindspore.float32) >>> result = net(input) >>> output = result.shape >>> print(output) (1, 240, 1024, 640)
-
class
tinyms.layers.DenseBnAct(in_channels, out_channels, weight_init='normal', bias_init='zeros', has_bias=True, has_bn=False, momentum=0.9, eps=1e-05, activation=None, alpha=0.2, after_fake=True)[source]¶ A combination of Dense, Batchnorm, and the activation layer.
This part is a more detailed overview of Dense op.
- Parameters
in_channels (int) – The number of channels in the input space.
out_channels (int) – The number of channels in the output space.
weight_init (Union[Tensor, str, Initializer, numbers.Number]) – The trainable weight_init parameter. The dtype is same as input. The values of str refer to the function initializer. Default: ‘normal’.
bias_init (Union[Tensor, str, Initializer, numbers.Number]) – The trainable bias_init parameter. The dtype is same as input. The values of str refer to the function initializer. Default: ‘zeros’.
has_bias (bool) – Specifies whether the layer uses a bias vector. Default: True.
has_bn (bool) – Specifies to use batchnorm or not. Default: False.
momentum (float) – Momentum for moving average for batchnorm, must be [0, 1]. Default:0.9
eps (float) – Term added to the denominator to improve numerical stability for batchnorm, should be greater than 0. Default: 1e-5.
activation (Union[str, Cell, Primitive]) – Specifies activation type. The optional values are as following: ‘softmax’, ‘logsoftmax’, ‘relu’, ‘relu6’, ‘tanh’, ‘gelu’, ‘sigmoid’, ‘prelu’, ‘leakyrelu’, ‘hswish’, ‘hsigmoid’. Default: None.
alpha (float) – Slope of the activation function at x < 0 for LeakyReLU. Default: 0.2.
after_fake (bool) – Determine whether there must be a fake quantization operation after DenseBnAct.
- Inputs:
input (Tensor) - Tensor of shape \((N, in\_channels)\).
- Outputs:
Tensor of shape \((N, out\_channels)\).
- Raises
TypeError – If in_channels or out_channels is not an int.
TypeError – If has_bias, has_bn or after_fake is not a bool.
TypeError – If momentum or eps is not a float.
ValueError – If momentum is not in range [0, 1.0].
- Supported Platforms:
AscendGPU
Examples
>>> net = nn.DenseBnAct(3, 4) >>> input = Tensor(np.random.randint(0, 255, [2, 3]), mindspore.float32) >>> result = net(input) >>> output = result.shape >>> print(output) (2, 4)
-
class
tinyms.layers.TimeDistributed(layer, time_axis, reshape_with_axis=None)[source]¶ The time distributed layer.
Time distributed is a wrapper which allows to apply a layer to every temporal slice of an input. And the input should be at least 3D. There are two cases in the implementation. When reshape_with_axis provided, the reshape method will be chosen, which is more efficient; otherwise, the method of dividing the inputs along time axis will be used, which is more general. For example, reshape_with_axis could not be provided when deal with batch normal.
- Parameters
- Inputs:
input (Tensor) - Tensor of shape \((N, T, *)\).
- Outputs:
Tensor of shape \((N, T, *)\)
- Supported Platforms:
AscendGPUCPU
- Raises
TypeError – If layer is not a Cell or Primitive.
Examples
>>> input = Tensor(np.random.random([32, 10, 3]), mindspore.float32) >>> dense = nn.Dense(3, 6) >>> net = nn.TimeDistributed(dense, time_axis=1, reshape_with_axis=0) >>> output = net(input) >>> print(output.shape) (32, 10, 6)
-
class
tinyms.layers.ForwardValueAndGrad(network, weights=None, get_all=False, get_by_list=False, sens_param=False)[source]¶ Network training package class.
Including the network and a gradient function. The resulting Cell is trained with input ‘*inputs’. The backward graph will be created in the gradient function to calculating gradient.
- Parameters
network (Cell) – The training network.
weights (ParameterTuple) – The parameters of the training network that need to calculate the gradient.
get_all (bool) – If True, get all the gradients with respect to inputs. Default: False.
get_by_list (bool) – If True, get all the gradients with respect to Parameter variables. If get_all and get_by_list are both False, get the gradient with respect to first input. If get_all and get_by_list are both True, get the gradients with respect to inputs and Parameter variables at the same time in the form of ((gradients with respect to inputs), (gradients with respect to parameters)). Default: False.
sens_param (bool) – Whether to append sensitivity (gradient with respect to output) as input. If sens_param is False, a ‘ones_like(outputs)’ sensitivity will be attached automatically. Default: False. If the sens_param is True, a sensitivity (gradient with respect to output) needs to be transferred through the input parameter.
- Inputs:
(*inputs) (Tuple(Tensor…)) - Tuple of inputs with shape \((N, \ldots)\).
- (sens) - A sensitivity (gradient with respect to output) as the input of backpropagation.
If network has single output, the sens is a tensor. If network has multiple outputs, the sens is the tuple(tensor).
- Outputs:
forward value - The result of network forward running.
gradients (tuple(tensor)) - The gradients of network parameters and inputs.
- Supported Platforms:
AscendGPUCPU
Examples
>>> class Net(nn.Cell): ... def __init__(self): ... super(Net, self).__init__() ... self.weight = Parameter(Tensor(np.ones([2, 2]).astype(np.float32)), name="weight") ... self.matmul = P.MatMul() ... ... def construct(self, x): ... out = self.matmul(x, self.weight) ... return out ... >>> net = Net() >>> criterion = nn.SoftmaxCrossEntropyWithLogits() >>> net_with_criterion = nn.WithLossCell(net, criterion) >>> weight = ParameterTuple(net.trainable_params()) >>> train_network = nn.ForwardValueAndGrad(net_with_criterion, weights=weight, get_all=True, get_by_list=True) >>> inputs = Tensor(np.ones([1, 2]).astype(np.float32)) >>> labels = Tensor(np.zeros([1, 2]).astype(np.float32)) >>> result = train_network(inputs, labels) >>> print(result) (Tensor(shape=[1], dtype=Float32, value=[0.00000000e+00]), ((Tensor(shape=[1, 2], dtype=Float32, value= [[1.00000000e+00, 1.00000000e+00]]), Tensor(shape=[1, 2], dtype=Float32, value= [[0.00000000e+00, 0.00000000e+00]])), (Tensor(shape=[2, 2], dtype=Float32, value= [[5.00000000e-01, 5.00000000e-01], [5.00000000e-01, 5.00000000e-01]]),)))
-
class
tinyms.layers.TrainOneStepCell(network, optimizer, sens=1.0)[source]¶ Network training package class.
Wraps the network with an optimizer. The resulting Cell is trained with input ‘*inputs’. The backward graph will be created in the construct function to update the parameter. Different parallel modes are available for training.
- Parameters
network (Cell) – The training network. The network only supports single output.
optimizer (Cell) – Optimizer for updating the weights.
sens (Number) – The scaling number to be filled as the input of backpropagation. Default value is 1.0.
- Inputs:
(*inputs) (Tuple(Tensor)) - Tuple of input tensors with shape \((N, \ldots)\).
- Outputs:
Tensor, a scalar Tensor with shape \(()\).
- Raises
TypeError – If sens is not a number.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> loss_fn = nn.SoftmaxCrossEntropyWithLogits() >>> optim = nn.Momentum(net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> #1) Using the WithLossCell existing provide >>> loss_net = nn.WithLossCell(net, loss_fn) >>> train_net = nn.TrainOneStepCell(loss_net, optim) >>> >>> #2) Using user-defined WithLossCell >>> class MyWithLossCell(Cell): ... def __init__(self, backbone, loss_fn): ... super(MyWithLossCell, self).__init__(auto_prefix=False) ... self._backbone = backbone ... self._loss_fn = loss_fn ... ... def construct(self, x, y, label): ... out = self._backbone(x, y) ... return self._loss_fn(out, label) ... ... @property ... def backbone_network(self): ... return self._backbone ... >>> loss_net = MyWithLossCell(net, loss_fn) >>> train_net = nn.TrainOneStepCell(loss_net, optim)
-
class
tinyms.layers.WithLossCell(backbone, loss_fn)[source]¶ Cell with loss function.
Wraps the network with loss function. This Cell accepts data and label as inputs and the computed loss will be returned.
- Parameters
backbone (Cell) – The target network to wrap.
loss_fn (Cell) – The loss function used to compute loss.
- Inputs:
data (Tensor) - Tensor of shape \((N, \ldots)\).
label (Tensor) - Tensor of shape \((N, \ldots)\).
- Outputs:
Tensor, a scalar tensor with shape \(()\).
- Raises
TypeError – If dtype of data or label is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> loss_fn = nn.SoftmaxCrossEntropyWithLogits(sparse=False) >>> net_with_criterion = nn.WithLossCell(net, loss_fn) >>> >>> batch_size = 2 >>> data = Tensor(np.ones([batch_size, 1, 32, 32]).astype(np.float32) * 0.01) >>> label = Tensor(np.ones([batch_size, 10]).astype(np.float32)) >>> >>> output_data = net_with_criterion(data, label)
-
property
backbone_network¶ Returns the backbone network.
- Returns
Cell, the backbone network.
-
class
tinyms.layers.WithGradCell(network, loss_fn=None, sens=None)[source]¶ Cell that returns the gradients.
Wraps the network with backward cell to compute gradients. A network with a loss function is necessary as argument. If loss function in None, the network must be a wrapper of network and loss function. This Cell accepts ‘*inputs’ as inputs and returns gradients for each trainable parameter.
Note
Run in PyNative mode.
- Parameters
network (Cell) – The target network to wrap. The network only supports single output.
loss_fn (Cell) – Primitive loss function used to compute gradients. Default: None.
sens (Union[None, Tensor, Scalar, Tuple ...]) – The sensitive for backpropagation, the type and shape must be same as the network output. If None, we will fill one to a same type shape of output value. Default: None.
- Inputs:
(*inputs) (Tuple(Tensor)) - Tuple of input tensors with shape \((N, \ldots)\).
- Outputs:
list, a list of Tensors with identical shapes as trainable weights.
- Raises
TypeError – If sens is not one of None, Tensor, Scalar or Tuple.
- Supported Platforms:
AscendGPU
Examples
>>> # For a defined network Net without loss function >>> net = Net() >>> loss_fn = nn.SoftmaxCrossEntropyWithLogits() >>> grad_net = nn.WithGradCell(net, loss_fn) >>> >>> # For a network wrapped with loss function >>> net = Net() >>> net_with_criterion = nn.WithLossCell(net, loss_fn) >>> grad_net = nn.WithGradCell(net_with_criterion)
-
class
tinyms.layers.WithEvalCell(network, loss_fn, add_cast_fp32=False)[source]¶ Cell that returns loss, output and label for evaluation.
This Cell accepts a network and loss function as arguments and computes loss for model. It returns loss, output and label to calculate the metrics.
- Parameters
network (Cell) – The network Cell.
loss_fn (Cell) – The loss Cell.
add_cast_fp32 (bool) – Adjust the data type to float32.
- Inputs:
data (Tensor) - Tensor of shape \((N, \ldots)\).
label (Tensor) - Tensor of shape \((N, \ldots)\).
- Outputs:
Tuple, containing a scalar loss Tensor, a network output Tensor of shape \((N, \ldots)\) and a label Tensor of shape \((N, \ldots)\).
- Raises
TypeError – If add_cast_fp32 is not a bool.
- Supported Platforms:
AscendGPU
Examples
>>> # For a defined network Net without loss function >>> net = Net() >>> loss_fn = nn.SoftmaxCrossEntropyWithLogits() >>> eval_net = nn.WithEvalCell(net, loss_fn)
-
class
tinyms.layers.GetNextSingleOp(dataset_types, dataset_shapes, queue_name)[source]¶ Cell to run for getting the next operation.
- Parameters
For detailed information, refer to ops.operations.GetNext.
- Supported Platforms:
AscendGPU
Examples
>>> train_dataset = create_custom_dataset() >>> dataset_helper = mindspore.DatasetHelper(train_dataset, dataset_sink_mode=True) >>> dataset = dataset_helper.iter.dataset >>> dataset_types, dataset_shapes = dataset_helper.types_shapes() >>> queue_name = dataset.__transfer_dataset__.queue_name >>> get_next_single_op_net = nn.GetNextSingleOp(dataset_types, dataset_shapes, queue_name) >>> data, label = get_next_single_op_net() >>> relu = P.ReLU() >>> result = relu(data).asnumpy() >>> print(result.shape) (32, 1, 32, 32)
-
class
tinyms.layers.TrainOneStepWithLossScaleCell(network, optimizer, scale_sense)[source]¶ Network training with loss scaling.
This is a training step with loss scaling. It takes a network, an optimizer and possibly a scale update Cell as args. The loss scale value can be updated in both host side or device side. The TrainOneStepWithLossScaleCell will be compiled to be graph which takes *inputs as input data. The Tensor type of scale_sense is acting as loss scaling value. If you want to update it on host side, the value must be provided. If the Tensor type of scale_sense is not given, the loss scale update logic must be provied by Cell type of scale_sense.
- Parameters
network (Cell) – The training network. The network only supports single output.
optimizer (Cell) – Optimizer for updating the weights.
scale_sense (Union[Tensor, Cell]) – If this value is Cell type, the loss scaling update logic cell.If this value is Tensor type, Tensor with shape \(()\) or \((1,)\).
- Inputs:
(*inputs) (Tuple(Tensor)) - Tuple of input tensors with shape \((N, \ldots)\).
- Outputs:
Tuple of 3 Tensor, the loss, overflow flag and current loss scaling value.
loss (Tensor) - Tensor with shape \(()\).
overflow (Tensor) - Tensor with shape \(()\), type is bool.
loss scaling value (Tensor) - Tensor with shape \(()\)
- Raises
TypeError – If scale_sense is neither Cell nor Tensor.
ValueError – If shape of scale_sense is neither (1,) nor ().
- Supported Platforms:
AscendGPU
Examples
>>> import numpy as np >>> from mindspore import Tensor, Parameter, nn >>> from mindspore.ops import operations as P >>> from mindspore.nn.wrap.cell_wrapper import WithLossCell >>> from mindspore.common import dtype as mstype >>> >>> class Net(nn.Cell): ... def __init__(self, in_features, out_features): ... super(Net, self).__init__() ... self.weight = Parameter(Tensor(np.ones([in_features, out_features]).astype(np.float32)), ... name='weight') ... self.matmul = P.MatMul() ... ... def construct(self, x): ... output = self.matmul(x, self.weight) ... return output ... >>> size, in_features, out_features = 16, 16, 10 >>> #1) when the type of scale_sense is Cell: >>> net = Net(in_features, out_features) >>> loss = nn.MSELoss() >>> optimizer = nn.Momentum(net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> net_with_loss = WithLossCell(net, loss) >>> manager = nn.DynamicLossScaleUpdateCell(loss_scale_value=2**12, scale_factor=2, scale_window=1000) >>> train_network = nn.TrainOneStepWithLossScaleCell(net_with_loss, optimizer, scale_sense=manager) >>> input = Tensor(np.ones([out_features, in_features]), mindspore.float32) >>> labels = Tensor(np.ones([out_features,]), mindspore.float32) >>> output = train_network(input, labels) >>> >>> #2) when the type of scale_sense is Tensor: >>> net = Net(in_features, out_features) >>> loss = nn.MSELoss() >>> optimizer = nn.Momentum(net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> net_with_loss = WithLossCell(net, loss) >>> inputs = Tensor(np.ones([size, in_features]).astype(np.float32)) >>> label = Tensor(np.zeros([size, out_features]).astype(np.float32)) >>> scaling_sens = Tensor(np.full((1), np.finfo(np.float32).max), dtype=mstype.float32) >>> train_network = nn.TrainOneStepWithLossScaleCell(net_with_loss, optimizer, scale_sense=scaling_sens) >>> output = train_network(inputs, label)
-
get_overflow_status(status, compute_output)[source]¶ Get floating-point overflow status.
Get overflow results after executing the target process for overflow detection.
- Parameters
status (object) – A status instance used to detect the overflow.
compute_output – Overflow detection should be performed on a certain computation. Set compute_output as the output of the computation, to ensure overflow status is acquired before executing the computation.
- Returns
bool, whether the overflow occurs or not.
-
process_loss_scale(overflow)[source]¶ Calculate loss scale according to the overflow.
- Parameters
overflow (bool) – Whether the overflow occurs or not.
- Returns
bool, overflow value.
-
set_sense_scale(sens)[source]¶ If the user has set the sens in the training process and wants to reassign the value, he can call this function again to make modification, and sens needs to be of type Tensor.
-
start_overflow_check(pre_cond, compute_input)[source]¶ Start floating-point overflow detection. Create and clear the overflow detection state.
Specify the argument ‘pre_cond’ and ‘compute_input’ to make sure overflow status is cleared at the right time. Taking this situation as an example, we need to execute state clearing after loss calculation and then detect overflow in the process of gradient calculation. In this case, pre_cond should be the output of the loss function, and compute_input should be the input of gradients-computing function.
- Parameters
pre_cond (object) – A precondition for starting overflow detection. It determines the executing order of overflow state clearing and prior processions. It makes sure that the function ‘start_overflow’ clears status after finishing the process of precondition.
compute_input (object) – The input of subsequent process. Overflow detection should be performed on a certain computation. Set compute_input as the input of the computation, to ensure overflow status is cleared before executing the computation.
- Returns
Tuple[object, object], the first value is False for GPU backend, while it is a instance of NPUAllocFloatStatus for other backend. The status is used to detect overflow during overflow detection. The second value is the same as the input of compute_input, but contains some information about the execution order.
-
class
tinyms.layers.DistributedGradReducer(parameters, mean=True, degree=None, fusion_type=1)[source]¶ A distributed optimizer.
Constructs a gradient reducer Cell, which applies communication and average operations on single-process gradient values.
- Parameters
parameters (list) – the parameters to be updated.
mean (bool) – When mean is true, the mean coefficient (degree) would apply on gradients. Default: False.
degree (int) – The mean coefficient. Usually it equals to device number. Default: None.
fusion_type (int) – The type of all reduce fusion. Default: 1.
- Raises
ValueError – If degree is not a int or less than 0.
- Supported Platforms:
Ascend,GPU
Examples
>>> # This example should be run with multiple processes. >>> # Please refer to the tutorial > Distributed Training on mindspore.cn. >>> import numpy as np >>> from mindspore.communication import init >>> from mindspore.ops import composite as C >>> from mindspore.ops import operations as P >>> from mindspore.ops import functional as F >>> from mindspore import context >>> from mindspore.context import ParallelMode >>> from mindspore import Parameter, Tensor >>> from mindspore import nn >>> from mindspore.nn.wrap.cell_wrapper import WithLossCell >>> from mindspore.parallel._utils import (_get_device_num, _get_gradients_mean) >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> init() >>> context.reset_auto_parallel_context() >>> context.set_auto_parallel_context(parallel_mode=ParallelMode.DATA_PARALLEL) >>> >>> class TrainingWrapper(nn.Cell): ... def __init__(self, network, optimizer, sens=1.0): ... super(TrainingWrapper, self).__init__(auto_prefix=False) ... self.network = network ... self.network.add_flags(defer_inline=True) ... self.weights = optimizer.parameters ... self.optimizer = optimizer ... self.grad = C.GradOperation(get_by_list=True, sens_param=True) ... self.sens = sens ... self.reducer_flag = False ... self.grad_reducer = None ... self.parallel_mode = context.get_auto_parallel_context("parallel_mode") ... if self.parallel_mode in [ParallelMode.DATA_PARALLEL, ParallelMode.HYBRID_PARALLEL]: ... self.reducer_flag = True ... if self.reducer_flag: ... mean = _get_gradients_mean() ... degree = _get_device_num() ... self.grad_reducer = nn.DistributedGradReducer(optimizer.parameters, mean, degree) ... ... def construct(self, *args): ... weights = self.weights ... loss = self.network(*args) ... sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens) ... grads = self.grad(self.network, weights)(*args, sens) ... if self.reducer_flag: ... # apply grad reducer on grads ... grads = self.grad_reducer(grads) ... return F.depend(loss, self.optimizer(grads)) >>> >>> class Net(nn.Cell): ... def __init__(self, in_features, out_features): ... super(Net, self).__init__() ... self.weight = Parameter(Tensor(np.ones([in_features, out_features]).astype(np.float32)), ... name='weight') ... self.matmul = P.MatMul() ... ... def construct(self, x): ... output = self.matmul(x, self.weight) ... return output >>> >>> size, in_features, out_features = 16, 16, 10 >>> network = Net(in_features, out_features) >>> loss = nn.MSELoss() >>> net_with_loss = WithLossCell(network, loss) >>> optimizer = nn.Momentum(net_with_loss.trainable_params(), learning_rate=0.1, momentum=0.9) >>> train_cell = TrainingWrapper(net_with_loss, optimizer) >>> inputs = Tensor(np.ones([size, in_features]).astype(np.float32)) >>> label = Tensor(np.zeros([size, out_features]).astype(np.float32)) >>> grads = train_cell(inputs, label) >>> print(grads) 256.0
-
class
tinyms.layers.ParameterUpdate(param)[source]¶ Cell that updates parameters.
With this Cell, one can manually update param with the input Tensor.
- Parameters
param (Parameter) – The parameter to be updated manually.
- Raises
KeyError – If parameter with the specified name does not exist.
- Supported Platforms:
Ascend
Examples
>>> network = nn.Dense(3, 4) >>> param = network.parameters_dict()['weight'] >>> update = nn.ParameterUpdate(param) >>> update.phase = "update_param" >>> weight = Tensor(np.arange(12).reshape((4, 3)), mindspore.float32) >>> output = update(weight)
-
class
tinyms.layers.DynamicLossScaleUpdateCell(loss_scale_value, scale_factor, scale_window)[source]¶ Dynamic Loss scale update cell.
For loss scaling training, the initial loss scaling value will be set to be loss_scale_value. In each training step, the loss scaling value will be updated by loss scaling value/scale_factor when there is an overflow. And it will be increased by loss scaling value * scale_factor if there is no overflow for a continuous scale_window steps. This cell is used for Graph mode training in which all logic will be executed on device side(Another training mode is normal(non-sink) mode in which some logic will be executed on host).
- Parameters
- Inputs:
inputs (Tensor) - Tensor of shape \((N, \ldots)\).
label (Tensor) - Tensor of shape \((N, \ldots)\).
- Outputs:
Tensor, a scalar Tensor with shape \(()\).
- Raises
TypeError – If dtype of inputs or label is neither float16 nor float32.
- Supported Platforms:
AscendGPU
Examples
>>> import numpy as np >>> from mindspore import Tensor, Parameter, nn >>> from mindspore.ops import operations as P >>> from mindspore.nn.wrap.cell_wrapper import WithLossCell >>> >>> class Net(nn.Cell): ... def __init__(self, in_features, out_features): ... super(Net, self).__init__() ... self.weight = Parameter(Tensor(np.ones([in_features, out_features]).astype(np.float32)), ... name='weight') ... self.matmul = P.MatMul() ... ... def construct(self, x): ... output = self.matmul(x, self.weight) ... return output ... >>> in_features, out_features = 16, 10 >>> net = Net(in_features, out_features) >>> loss = nn.MSELoss() >>> optimizer = nn.Momentum(net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> net_with_loss = WithLossCell(net, loss) >>> manager = nn.DynamicLossScaleUpdateCell(loss_scale_value=2**12, scale_factor=2, scale_window=1000) >>> train_network = nn.TrainOneStepWithLossScaleCell(net_with_loss, optimizer, scale_sense=manager) >>> input = Tensor(np.ones([out_features, in_features]), mindspore.float32) >>> labels = Tensor(np.ones([out_features,]), mindspore.float32) >>> output = train_network(input, labels)
-
class
tinyms.layers.FixedLossScaleUpdateCell(loss_scale_value)[source]¶ Static scale update cell, the loss scaling value will not be updated.
For usage, refer to DynamicLossScaleUpdateCell.
- Parameters
loss_scale_value (float) – Initializes loss scale.
- Supported Platforms:
AscendGPU
Examples
>>> import numpy as np >>> from mindspore import Tensor, Parameter, nn >>> from mindspore.ops import operations as P >>> from mindspore.nn.wrap.cell_wrapper import WithLossCell >>> >>> class Net(nn.Cell): ... def __init__(self, in_features, out_features): ... super(Net, self).__init__() ... self.weight = Parameter(Tensor(np.ones([in_features, out_features]).astype(np.float32)), ... name='weight') ... self.matmul = P.MatMul() ... ... def construct(self, x): ... output = self.matmul(x, self.weight) ... return output ... >>> in_features, out_features = 16, 10 >>> net = Net(in_features, out_features) >>> loss = nn.MSELoss() >>> optimizer = nn.Momentum(net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> net_with_loss = WithLossCell(net, loss) >>> manager = nn.FixedLossScaleUpdateCell(loss_scale_value=2**12) >>> train_network = nn.TrainOneStepWithLossScaleCell(net_with_loss, optimizer, scale_sense=manager) >>> input = Tensor(np.ones([out_features, in_features]), mindspore.float32) >>> labels = Tensor(np.ones([out_features,]), mindspore.float32) >>> output = train_network(input, labels)
-
class
tinyms.layers.VirtualDatasetCellTriple(backbone)[source]¶ Wrap the network with virtual dataset to convert data parallel layout to model parallel layout.
VirtualDatasetCellTriple is a virtual Primitive, it does not exist in the final executing graph. Inputs and outputs of VirtualDatasetCellTriple are distributed in data parallel pattern, tensor redistribution Primitives is inserted dynamically during the graph compile process.
Note
Only used in semi auto parallel and auto parallel mode. There are three inputs, as contrary to two inputs in _VirtualDatasetCell.
- Parameters
backbone (Cell) – The target network to wrap.
Examples
>>> net = Net() >>> net = VirtualDatasetCellTriple(net)
tinyms.model¶
-
class
tinyms.model.Model(network)[source]¶ High-Level API for Training or Evaluation.
Model groups layers into an object with training and inference features.
- Parameters
network (layers.Layer) – A training or testing network.
Examples
>>> from tinyms.model import Model, lenet5 >>> form tinyms.losses import SoftmaxCrossEntropyWithLogits >>> from tinyms.optimizers import Momentum >>> >>> net = lenet5(class_num=10) >>> model = Model(net) >>> net_loss = SoftmaxCrossEntropyWithLogits() >>> net_opt = Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> model.compile(loss_fn=net_loss, optimizer=net_opt, metrics=None) >>> # For details about how to build the dataset, please refer to the API document on the official website. >>> ds_train = create_custom_dataset() >>> model.train(2, ds_train)
-
compile(loss_fn=None, optimizer=None, metrics=None, eval_network=None, amp_level='O0', **kwargs)[source]¶ High-Level API for configure the train or eval network.
- Parameters
loss_fn (layers.Layer) – Objective function, if loss_fn is None, the network should contain the logic of loss and grads calculation, and the logic of parallel if needed. Default: None.
optimizer (layers.Layer) – Optimizer for updating the weights. Default: None.
metrics (Union[dict, set]) – A Dictionary or a set of metrics to be evaluated by the model during training and testing. eg: {‘accuracy’, ‘recall’}. Default: None.
eval_network (layers.Layer) – Network for evaluation. If not defined, network and loss_fn would be wrapped as eval_network. Default: None.
amp_level (str) –
Option for argument level in mindspore.amp.build_train_network, level for mixed precision training. Supports [“O0”, “O2”, “O3”, “auto”]. Default: “O0”.
O0: Do not change.
O2: Cast network to float16, keep batchnorm run in float32, using dynamic loss scale.
O3: Cast network to float16, with additional property ‘keep_batchnorm_fp32=False’.
auto: Set to level to recommended level in different devices. Set level to O2 on GPU, Set
level to O3 Ascend. The recommended level is choose by the export experience, cannot always generalize. User should specify the level for special network.
O2 is recommended on GPU, O3 is recommended on Ascend.
-
eval(valid_dataset, callbacks=None, dataset_sink_mode=True)[source]¶ Evaluation API where the iteration is controlled by python front-end.
Configure to pynative mode or CPU, the evaluating process will be performed with dataset non-sink mode.
Note
If dataset_sink_mode is True, data will be sent to device. If device is Ascend, features of data will be transferred one by one. The limitation of data transmission per time is 256M.
- Parameters
- Returns
Dict, which returns the loss value and metrics values for the model in the test mode.
Examples
>>> from mindspore import Model, nn >>> >>> # For details about how to build the dataset, please refer to the tutorial >>> # document on the official website. >>> dataset = create_custom_dataset() >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=None, metrics={'acc'}) >>> acc = model.eval(dataset, dataset_sink_mode=False)
-
property
eval_network¶ Get the model’s eval_network.
-
export(inputs, file_name, file_format='MINDIR', **kwargs)[source]¶ Export the TinyMS prediction model to a file in the specified format.
- Parameters
inputs (Tensor) – Inputs of the net.
file_name (str) – File name of the model to be exported.
file_format (str) –
MindSpore currently supports AIR, ONNX and MINDIR format for exported model. Default: MINDIR.
AIR: Ascend Intermediate Representation. An intermediate representation format of Ascend model.
Recommended suffix for output file is ‘.air’.
ONNX: Open Neural Network eXchange. An open format built to represent machine learning models.
Recommended suffix for output file is ‘.onnx’.
MINDIR: MindSpore Native Intermediate Representation for Anf. An intermediate representation format
for MindSpore models. Recommended suffix for output file is ‘.mindir’.
kwargs (dict) –
Configuration options dictionary.
quant_mode: The mode of quant.
mean: Input data mean. Default: 127.5.
std_dev: Input data variance. Default: 127.5.
-
infer_predict_layout(*predict_data)[source]¶ Generate parameter layout for the predict network in auto or semi auto parallel mode.
Data could be a single tensor or multiple tensors.
Note
Batch data should be put together in one tensor.
- Parameters
predict_data (Tensor) – One tensor or multiple tensors of predict data.
- Returns
Dict, Parameter layout dictionary used for load distributed checkpoint
Examples
>>> import numpy as np >>> import mindspore as ms >>> from mindspore import Model, context, Tensor >>> from mindspore.context import ParallelMode >>> >>> context.set_context(mode=context.GRAPH_MODE) >>> context.set_auto_parallel_context(full_batch=True, parallel_mode=ParallelMode.SEMI_AUTO_PARALLEL) >>> input_data = Tensor(np.random.randint(0, 255, [1, 3, 224, 224]), ms.float32) >>> model = Model(Net()) >>> model.infer_predict_layout(input_data)
-
load_checkpoint(ckpt_file_name, strict_load=False)[source]¶ Loads checkpoint info from a specified file.
- Parameters
- Returns
Dict, key is parameter name, value is a Parameter.
- Raises
ValueError – Checkpoint file is incorrect.
Examples
>>> ckpt_file_name = "./checkpoint/LeNet5-1_32.ckpt" >>> param_dict = model.load_checkpoint(ckpt_file_name)
-
predict(*predict_data)[source]¶ Generate output predictions for the input samples.
Data could be a single tensor, a list of tensor, or a tuple of tensor.
Note
Batch data should be put together in one tensor.
- Parameters
predict_data (Tensor) – The predict data, can be bool, int, float, str, None, tensor, or tuple, list and dict that store these types.
- Returns
Tensor, array(s) of predictions.
Examples
>>> import mindspore as ms >>> from mindspore import Model, Tensor >>> >>> input_data = Tensor(np.random.randint(0, 255, [1, 1, 32, 32]), ms.float32) >>> model = Model(Net()) >>> result = model.predict(input_data)
-
property
predict_network¶ Get the model’s predict_network.
-
train(epoch, train_dataset, callbacks=None, dataset_sink_mode=True, sink_size=-1)[source]¶ Training API where the iteration is controlled by python front-end.
When setting pynative mode or CPU, the training process will be performed with dataset not sink.
Note
If dataset_sink_mode is True, data will be sent to device. If device is Ascend, features of data will be transferred one by one. The limitation of data transmission per time is 256M. If sink_size > 0, each epoch the dataset can be traversed unlimited times until you get sink_size elements of the dataset. Next epoch continues to traverse from the end position of the previous traversal.
- Parameters
epoch (int) – Generally, total number of iterations on the data per epoch. When dataset_sink_mode is set to true and sink_size>0, each epoch sink sink_size steps on the data instead of total number of iterations.
train_dataset (Dataset) – A training dataset iterator. If there is no loss_fn, a tuple with multiple data (data1, data2, data3, …) should be returned and passed to the network. Otherwise, a tuple (data, label) should be returned. The data and label would be passed to the network and loss function respectively.
callbacks (Optional[list[Callback], Callback]) – List of callback objects or callback object, which should be executed while training. Default: None.
dataset_sink_mode (bool) – Determines whether to pass the data through dataset channel. Default: True. Configure pynative mode or CPU, the training process will be performed with dataset not sink. Default: True.
sink_size (int) – Control the amount of data in each sink. If sink_size = -1, sink the complete dataset for each epoch. If sink_size > 0, sink sink_size data for each epoch. If dataset_sink_mode is False, set sink_size as invalid. Default: -1.
Examples
>>> from mindspore import Model, nn >>> from mindspore.train.loss_scale_manager import FixedLossScaleManager >>> >>> # For details about how to build the dataset, please refer to the tutorial >>> # document on the official website. >>> dataset = create_custom_dataset() >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> loss_scale_manager = FixedLossScaleManager() >>> optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None, loss_scale_manager=loss_scale_manager) >>> model.train(2, dataset)
-
property
train_network¶ Get the model’s train_network.
-
tinyms.model.load(file_name)[source]¶ Load MindIR graph and return the network with parameters.
The returned object is wrapperred by a GraphLayer. However, there are some limitations to the current use of GraphLayer, see class
tinyms.layers.GraphLayerfor more details.- Parameters
file_name (str) – MindIR file name.
- Returns
GraphLayer instance, a compiled graph with parameters.
- Raises
ValueError – MindIR file is incorrect.
Examples
>>> import tinyms as ts >>> import tinyms.layers as layers >>> from tinyms.model import Model, load >>> >>> net = layers.Conv2d(1, 1, kernel_size=3) >>> model = Model(net) >>> input = ts.ones([1, 1, 3, 3]) >>> model.export(input, "net", file_format="MINDIR") ... >>> net = load("net.mindir") >>> print(net(input)) [[[[ 0.02548009 0.04010789 0.03120251] [ 0.00268656 0.02353744 0.03807815] [-0.00896441 -0.00303641 0.01502199]]]]
-
tinyms.model.lenet5(**kwargs)[source]¶ Get LeNet5 neural network.
- Parameters
class_num (int) – Class number. Default: 10.
- Returns
layers.Layer, layer instance of LeNet5 neural network.
Examples
>>> net = lenet5(class_num=10)
-
class
tinyms.model.LeNet(class_num=10, channel_num=1)[source]¶ LeNet architecture.
- Parameters
- Returns
Tensor, output tensor.
Examples
>>> LeNet(class_num=10)
-
tinyms.model.resnet50(**kwargs)[source]¶ Get ResNet50 neural network.
- Parameters
class_num (int) – Class number. Default: 10.
- Returns
layers.Layer, layer instance of ResNet50 neural network.
Examples
>>> net = resnet50(10)
-
class
tinyms.model.ResNet(block, layer_nums, in_channels, out_channels, strides, num_classes)[source]¶ ResNet architecture.
- Parameters
block (layers.Layer) – Block for network.
layer_nums (list) – Numbers of block in different layers.
in_channels (list) – Input channel in each layer.
out_channels (list) – Output channel in each layer.
strides (list) – Stride size in each layer.
num_classes (int) – The number of classes that the training images are belonging to.
- Returns
Tensor, output tensor.
Examples
>>> ResNet(ResidualBlock, >>> [3, 4, 6, 3], >>> [64, 256, 512, 1024], >>> [256, 512, 1024, 2048], >>> [1, 2, 2, 2], >>> 10)
-
tinyms.model.mobilenetv2(**kwargs)[source]¶ Get MobileNetV2 instance for model training, evaluation and prediction.
-
class
tinyms.model.MobileNetV2(class_num=1000, width_mult=1.0, round_nearest=8, input_channel=32, last_channel=1280, is_training=True)[source]¶ MobileNetV2 architecture.
- Parameters
- Returns
Tensor, output tensor.
-
tinyms.model.ssd300_mobilenetv2(**kwargs)[source]¶ Get SSD300 model instance for training, evaluation and prediction.
-
class
tinyms.model.SSD300(backbone, class_num=21, is_training=True)[source]¶ SSD300 Network. Default backbone is MobileNetV2.
- Parameters
backbone (layers.Layer) – backbone of ssd300 model.
class_num (int) – number of classes. Default: 21.
is_training (bool) – Specify if in training step. Default: True.
- Returns
Tensor, localization predictions. Tensor, class conf scores.
-
tinyms.model.cycle_gan(G_A, G_B)[source]¶ Get Cycle GAN network.
- Parameters
G_A (layers.Layer) – The generator net, currently it should be in [resnet, unet].
G_B (layers.Layer) – The generator net, currently it should be in [resnet, unet].
- Returns
Cycle GAN instance.
Examples
>>> gan_net = cycle_gan(G_A, G_B)
-
tinyms.model.cycle_gan_infer(g_model='resnet')[source]¶ Get Cycle GAN network for predict.
- Parameters
g_model (str) – The generator network type, currently it should be in [‘resnet’, ‘unet’]. Default: resnet.
- Returns
Cycle GAN instance.
Examples
>>> gan_net = cycle_gan(G_A, G_B)
-
tinyms.model.densenet100(**kwargs)[source]¶ Get DenseNet instance for model training, evaluation and prediction.
- Parameters
class_num (int) – The number of classes. Default: 10.
- Returns
model.DenseNet, DenseNet instance.
-
tinyms.model.alexnet(**kwargs)[source]¶ Get AlexNet neural network.
- Parameters
class_num (int) – Class number. Default: 10.
- Returns
layers.Layer, layer instance of AlexNet neural network.
Examples
>>> from tinyms.model import alexnet >>> >>> net = alexnet(class_num=10)
-
class
tinyms.model.AlexNet(class_num=1000)[source]¶ Get AlexNet neural network.
- Parameters
class_num (int) – Class number. Default: 1000.
- Returns
layers.Layer, layer instance of AlexNet neural network.
Examples
>>> from tinyms.model import AlexNet >>> >>> net = AlexNet(class_num=1000)
-
tinyms.model.sentimentnet(vocab_size, embed_size, num_hiddens, num_layers, bidirectional, num_classes, weight, batch_size)[source]¶ Sentiment network structure.
-
class
tinyms.model.SentimentNet(vocab_size, embed_size, num_hiddens, num_layers, bidirectional, num_classes, weight, batch_size)[source]¶ Sentiment network structure.
-
tinyms.model.bert(config, is_training, use_one_hot_embeddings=False)[source]¶ Get bert neural network.
-
class
tinyms.model.Bert(config, is_training, use_one_hot_embeddings=False)[source]¶ Bidirectional Encoder Representations from Transformers.
- Parameters
-
tinyms.model.vgg11(**kwargs)[source]¶ Get vgg11 neural network.
- Parameters
- Returns
layers.Layer, layer instance of vgg11 neural network.
Examples
>>> from tinyms.model import vgg11 >>> >>> net = vgg11(class_num=10)
-
tinyms.model.vgg13(**kwargs)[source]¶ Get vgg13 neural network.
- Parameters
- Returns
layers.Layer, layer instance of vgg13 neural network.
Examples
>>> from tinyms.model import vgg13 >>> >>> net = vgg13(class_num=10)
-
tinyms.model.vgg16(**kwargs)[source]¶ Get vgg16 neural network.
- Parameters
- Returns
layers.Layer, layer instance of vgg16 neural network.
Examples
>>> from tinyms.model import vgg16 >>> >>> net = vgg16(class_num=10)
-
tinyms.model.vgg19(**kwargs)[source]¶ Get vgg19 neural network.
- Parameters
- Returns
layers.Layer, layer instance of vgg19 neural network.
Examples
>>> from tinyms.model import vgg19 >>> >>> net = vgg19(class_num=10)
-
class
tinyms.model.VGG(features, class_num=1000)[source]¶ Get VGG neural network.
- Parameters
features (layers.Layer) – Feature extractor.
class_num (int) – Class number. Default: 1000.
- Returns
layers.Layer, layer instance of AlexNet neural network.
Examples
>>> from tinyms.model import VGG >>> >>> net = VGG(features=make_layers(cfg=cfgs['A']),class_num=1000)
tinyms.initializers¶
-
class
tinyms.initializers.Initializer(**kwargs)[source]¶ The base class of the initializer. Initialization of tensor basic attributes and model weight values.
- Parameters
kwargs (dict) – Keyword arguments for Initializer.
- Returns
Array, an array after being initialized.
-
tinyms.initializers.initializer(init, shape=None, dtype=mindspore.float32)[source]¶ Create and initialize a tensor.
- Parameters
init (Union[Tensor, str, Initializer, numbers.Number]) –
Initialize value.
str: The init should be the alias of the class inheriting from Initializer and the corresponding class will be called.
Initializer: The init should be the class inheriting from Initializer to initialize tensor.
numbers.Number: The Constant will be called to initialize tensor.
shape (Union[tuple, list, int]) – A list of integers, a tuple of integers or an integer as the shape of output. Default: None.
dtype (
mindspore.dtype) – The type of data in initialized tensor. Default: mindspore.float32.
- Returns
Union[Tensor], return is Tensor object.
Examples
>>> import mindspore >>> from mindspore.common.initializer import initializer, One >>> tensor = initializer('ones', [1, 2, 3], mindspore.float32) >>> tensor = initializer(One(), [1, 2, 3], mindspore.float32) >>> tensor = initializer(0, [1, 2, 3], mindspore.float32)
-
class
tinyms.initializers.TruncatedNormal(sigma=0.01)[source]¶ Initialize a truncated normal distribution which is a bounded normal distribution within N(low, high).
- Parameters
sigma (float) – The sigma of the array. Default: 0.01.
- Returns
Array, truncated normal array.
-
class
tinyms.initializers.Normal(sigma=0.01)[source]¶ Initialize a normal array, and obtain values N(0, sigma) from the uniform distribution to fill the input tensor.
- Parameters
sigma (float) – The sigma of the array. Default: 0.01.
- Returns
Array, normal array.
-
class
tinyms.initializers.Uniform(scale=0.07)[source]¶ Initialize a uniform array, and obtain values U(-scale, scale) from the uniform distribution to fill the input tensor.
- Parameters
scale (float) – The scale of the array. Default: 0.07.
- Returns
Array, uniform array.
-
class
tinyms.initializers.HeUniform(negative_slope=0, mode='fan_in', nonlinearity='leaky_relu')[source]¶ Initialize the array with He kaiming uniform algorithm, and from a uniform distribution collect samples within U[-boundary, boundary] The boundary is defined as:
\[boundary = \sqrt{\frac{6}{(1 + a^2) \times \text{fan_in}}}\]- Parameters
negative_slope (int, float, bool) – The negativa slope of the rectifier used after this layer (only used when nonlinearity is ‘leaky_relu’). Default: 0.
mode (str) – Either ‘fan_in’ or ‘fan_out’. Choosing ‘fan_in’ preserves the magnitude of the variance of the weights in the forward pass. Choosing ‘fan_out’ preserves the magnitudes in the backwards pass. Default: fan_in.
nonlinearity (str) – The non-linear function, recommended to use only with ‘relu’ or ‘leaky_relu’. Default: leaky_relu.
- Returns
Array, assigned array.
-
class
tinyms.initializers.HeNormal(negative_slope=0, mode='fan_in', nonlinearity='leaky_relu')[source]¶ Initialize the array with He kaiming Normal algorithm, and from a normal distribution collect samples within N(0, sigma).
- Parameters
negative_slope (int, float, bool) – The negativa slope of the rectifier used after this layer (only used when nonlinearity is ‘leaky_relu’). Default: 0.
mode (str) – Either ‘fan_in’ or ‘fan_out’. Choosing ‘fan_in’ preserves the magnitude of the variance of the weights in the forward pass. Choosing ‘fan_out’ preserves the magnitudes in the backwards pass. Default: fan_in.
nonlinearity (str) – The non-linear function, recommended to use only with ‘relu’ or ‘leaky_relu’. Default: leaky_relu.
- Returns
Array, assigned array.
-
class
tinyms.initializers.XavierUniform(gain=1)[source]¶ Initialize the array with xavier uniform algorithm, and from a uniform distribution collect samples within U[-boundary, boundary] The boundary is defined as:
\[boundary = gain * \sqrt{\frac{6}{n_{in} + n_{out}}}\]where \(n_{in}\) is the number of input units in the weight tensor.
where \(n_{out}\) is the number of output units in the weight tensor.
- Parameters
gain (float) – An optional scaling factor. Default: 1.
- Returns
Array, assigned array.
-
class
tinyms.initializers.One(**kwargs)[source]¶ Initialize the array to one.
- Parameters
arr (Array) – The array to be assigned.
- Returns
Array, assigned array.
-
class
tinyms.initializers.Zero(**kwargs)[source]¶ Initialize the array to zero.
- Parameters
arr (Array) – The array to be assigned.
- Returns
Array, an array after being assigned.
-
class
tinyms.initializers.Constant(value)[source]¶ Initialize a constant.
- Parameters
value (Union[int, numpy.ndarray]) – The value to initialize.
- Returns
Array, an array after being assigned.
tinyms.losses¶
Losses module. Loss function in machine learning is the target of the model. It shows how well the model works on a dataset and the optimization target which the optimizer is searching.
-
tinyms.losses.net_with_loss(net)[source]¶ This function is provided for AI beginners who are not familiar with which loss should be chosen for the network to be trained. Instead of choosing different loss function, users could directly get the best suitable loss function by specifying network.
- Parameters
net (layers.Layer) – The instance of network to be trained.
- Raises
TypeError – When network type is not supported.
Note
Currently this function only supports few networks, if the network type is not supported, the system would raise TypeError exception.
Examples
>>> from tinyms.model import ssd300 >>> from tinyms.losses import net_with_loss >>> >>> net = ssd300() >>> net_loss = net_with_loss(net)
-
class
tinyms.losses.SSD300WithLoss(network)[source]¶ Provide SSD300 training loss through network.
- Parameters
network (layers.Layer) – The training network.
- Returns
Tensor, the loss of the network.
Examples
>>> from tinyms.model import ssd300 >>> from tinyms.losses import SSD300WithLoss >>> >>> net = SSD300WithLoss(ssd300())
-
class
tinyms.losses.CrossEntropyWithLabelSmooth(smooth_factor=0.0, num_classes=1000)[source]¶ CrossEntropyWith LabelSmooth.
- Parameters
- Returns
None.
Examples
>>> CrossEntropyWithLabelSmooth(smooth_factor=0., num_classes=1000)
-
class
tinyms.losses.CycleGANGeneratorLoss(generator, D_A, D_B)[source]¶ Cycle GAN generator loss.
- Parameters
generator (layers.Layer) – Generator of CycleGAN.
D_A (layers.Layer) – The discriminator network of domain A to domain B.
D_B (layers.Layer) – The discriminator network of domain B to domain A.
- Outputs:
Tuple Tensor, the losses of generator.
-
class
tinyms.losses.CycleGANDiscriminatorLoss(D_A, D_B, reduction='none')[source]¶ Cycle GAN discriminator loss.
- Parameters
D_A (layers.Layer) – The discriminator network of domain A to domain B.
D_B (layers.Layer) – The discriminator network of domain B to domain A.
reduction (str) – The discriminator network of reduction. Default: none.
- Outputs:
the loss of discriminator.
-
class
tinyms.losses.L1Loss(reduction='mean')[source]¶ L1Loss creates a criterion to measure the mean absolute error (MAE) between \(x\) and \(y\) element-wise, where \(x\) is the input Tensor and \(y\) is the target Tensor.
For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:
\[L(x, y) = \{l_1,\dots,l_N\}, \quad \text{with } l_n = \left| x_n - y_n \right|\]When argument reduction is ‘mean’, the mean value of \(L(x, y)\) will be returned. When argument reduction is ‘sum’, the sum of \(L(x, y)\) will be returned. \(N\) is the batch size.
- Parameters
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.
- Inputs:
input_data (Tensor) - Tensor of shape \((x_1, x_2, ..., x_R)\).
target_data (Tensor) - Tensor of shape \((y_1, y_2, ..., y_S)\).
- Outputs:
Tensor, loss float tensor.
- Raises
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = nn.L1Loss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) 0.33333334
-
class
tinyms.losses.MSELoss(reduction='mean')[source]¶ MSELoss creates a criterion to measure the mean squared error (squared L2-norm) between \(x\) and \(y\) element-wise, where \(x\) is the input and \(y\) is the target.
For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:
\[L(x, y) = \{l_1,\dots,l_N\}, \quad \text{with} \quad l_n = (x_n - y_n)^2.\]When argument reduction is ‘mean’, the mean value of \(L(x, y)\) will be returned. When argument reduction is ‘sum’, the sum of \(L(x, y)\) will be returned. \(N\) is the batch size.
- Parameters
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.
- Inputs:
input_data (Tensor) - Tensor of shape \((x_1, x_2, ..., x_R)\).
target_data (Tensor) - Tensor of shape \((y_1, y_2, ..., y_S)\).
- Outputs:
Tensor, weighted loss float tensor.
- Raises
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = nn.MSELoss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) 0.33333334
-
class
tinyms.losses.SmoothL1Loss(beta=1.0)[source]¶ A loss class for learning region proposals.
SmoothL1Loss can be regarded as modified version of L1Loss or a combination of L1Loss and L2Loss. L1Loss computes the element-wise absolute difference between two input Tensor while L2Loss computes the squared difference between two input Tensor. L2Loss often leads to faster convergence but it is less robust to outliers.
Given two input \(x,\ y\) of length \(N\), the unreduced SmoothL1Loss can be described as follows:
\[\begin{split}L_{i} = \begin{cases} \frac{0.5 (x_i - y_i)^{2}}{\text{beta}}, & \text{if } |x_i - y_i| < \text{beta} \\ |x_i - y_i| - 0.5 \text{beta}, & \text{otherwise. } \end{cases}\end{split}\]Here \(\text{beta}\) controls the point where the loss function changes from quadratic to linear. Its default value is 1.0. \(N\) is the batch size. This function returns an unreduced loss Tensor.
- Parameters
beta (float) – A parameter used to control the point where the function will change from quadratic to linear. Default: 1.0.
- Inputs:
input_data (Tensor) - Tensor of shape \((x_1, x_2, ..., x_R)\). Data type must be float16 or float32.
target_data (Tensor) - Ground truth data, with the same type and shape as input_data.
- Outputs:
Tensor, loss float tensor.
- Raises
TypeError – If beta is not a float.
TypeError – If dtype of input_data or target_data is neither float16 not float32.
ValueError – If beta is less than or equal to 0.
ValueError – If shape of input_data is not the same as target_data.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = nn.SmoothL1Loss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) [0. 0. 0.5]
-
class
tinyms.losses.FocalLoss(weight=None, gamma=2.0, reduction='mean')[source]¶ The loss function proposed by Kaiming team in their paper
Focal Loss for Dense Object Detectionimproves the effect of image object detection. It is a loss function to solve the imbalance of categories and the difference of classification difficulty. If you want to learn more, please refer to the paper. https://arxiv.org/pdf/1708.02002.pdf. The function is shown as follows:\[FL(p_t) = -(1-p_t)^\gamma log(p_t)\]- Parameters
gamma (float) – Gamma is used to adjust the steepness of weight curve in focal loss. Default: 2.0.
weight (Union[Tensor, None]) – A rescaling weight applied to the loss of each batch element. The dimension of weight should be 1. If None, no weights are applied. Default: None.
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. If “none”, do not perform reduction. Default: “mean”.
- Inputs:
predict (Tensor) - Tensor of shape should be (B, C) or (B, C, H) or (B, C, H, W). Where C is the number of classes. Its value is greater than 1. If the shape is (B, C, H, W) or (B, C, H), the H or product of H and W should be the same as target.
target (Tensor) - Tensor of shape should be (B, C) or (B, C, H) or (B, C, H, W). The value of C is 1 or it needs to be the same as predict’s C. If C is not 1, the shape of target should be the same as that of predict, where C is the number of classes. If the shape is (B, C, H, W) or (B, C, H), the H or product of H and W should be the same as predict.
- Outputs:
Tensor, it’s a tensor with the same shape and type as input predict.
- Raises
TypeError – If the data type of
gammais not float..TypeError – If
weightis not a Tensor.ValueError – If
targetdim different frompredict.ValueError – If
targetchannel is not 1 andtargetshape is different frompredict.ValueError – If
reductionis not one of ‘none’, ‘mean’, ‘sum’.
- Supported Platforms:
AscendGPU
Example
>>> predict = Tensor([[0.8, 1.4], [0.5, 0.9], [1.2, 0.9]], mstype.float32) >>> target = Tensor([[1], [1], [0]], mstype.int32) >>> focalloss = nn.FocalLoss(weight=Tensor([1, 2]), gamma=2.0, reduction='mean') >>> output = focalloss(predict, target) >>> print(output) 0.12516622
-
class
tinyms.losses.SoftmaxCrossEntropyWithLogits(sparse=False, reduction='none')[source]¶ Computes softmax cross entropy between logits and labels.
Measures the distribution error between the probabilities of the input (computed with softmax function) and the target where the classes are mutually exclusive (only one class is positive) using cross entropy loss.
Typical input into this function is unnormalized scores denoted as x whose shape is (N, C), and the corresponding targets.
For each instance \(x_i\), i ranges from 0 to N-1, the loss is given as:
\[\ell(x_i, c) = - \log\left(\frac{\exp(x_i[c])}{\sum_j \exp(x_i[j])}\right) = -x_i[c] + \log\left(\sum_j \exp(x_i[j])\right)\]where \(x_i\) is a 1D score Tensor, \(c\) is the index of 1 in one-hot.
Note
While the target classes are mutually exclusive, i.e., only one class is positive in the target, the predicted probabilities need not to be exclusive. It is only required that the predicted probability distribution of entry is a valid one.
- Parameters
- Inputs:
logits (Tensor) - Tensor of shape (N, C). Data type must be float16 or float32.
labels (Tensor) - Tensor of shape (N, ). If sparse is True, The type of labels is int32 or int64. If sparse is False, the type of labels is the same as the type of logits.
- Outputs:
Tensor, a tensor of the same shape and type as logits with the component-wise logistic losses.
- Raises
TypeError – If sparse is not a bool.
TypeError – If sparse is True and dtype of labels is neither int32 not int64.
TypeError – If sparse is False and dtype of labels is neither float16 not float32.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True) >>> np.random.seed(0) >>> logits = Tensor(np.random.randint(0, 9, [1, 10]), mindspore.float32) >>> labels_np = np.ones([1,]).astype(np.int32) >>> labels = Tensor(labels_np) >>> output = loss(logits, labels) >>> print(output) [7.868383]
-
class
tinyms.losses.BCELoss(weight=None, reduction='none')[source]¶ BCELoss creates a criterion to measure the binary cross entropy between the true labels and predicted labels.
Set the predicted labels as \(x\), true labels as \(y\), the output loss as \(\ell(x, y)\). Let,
\[L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log x_n + (1 - y_n) \cdot \log (1 - x_n) \right]\]Then,
\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]Note
Note that the predicted labels should always be the output of sigmoid and the true labels should be numbers between 0 and 1.
- Parameters
- Inputs:
inputs (Tensor) - The input Tensor. The data type must be float16 or float32.
labels (Tensor) - The label Tensor which has same shape and data type as inputs.
- Outputs:
Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as inputs. Otherwise, the output is a scalar.
- Raises
TypeError – If dtype of inputs, labels or weight (if given) is neither float16 not float32.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
ValueError – If shape of inputs is not the same as labels or weight (if given).
- Supported Platforms:
AscendGPUCPU
Examples
>>> weight = Tensor(np.array([[1.0, 2.0, 3.0], [4.0, 3.3, 2.2]]), mindspore.float32) >>> loss = nn.BCELoss(weight=weight, reduction='mean') >>> inputs = Tensor(np.array([[0.1, 0.2, 0.3], [0.5, 0.7, 0.9]]), mindspore.float32) >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]), mindspore.float32) >>> output = loss(inputs, labels) >>> print(output) 1.8952923
-
class
tinyms.losses.BCEWithLogitsLoss(reduction='mean', weight=None, pos_weight=None)[source]¶ Adds sigmoid activation function to input predict, and uses the given logits to compute binary cross entropy between the target and the output.
Sets input predict as X, input target as Y, output as L. Then,
\[p_{ij} = sigmoid(X_{ij}) = \frac{1}{1 + e^{-X_{ij}}}\]\[L_{ij} = -[Y_{ij} * ln(p_{ij}) + (1 - Y_{ij})ln(1 - p_{ij})]\]Then,
\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]- Parameters
reduction (str) – Type of reduction to be applied to loss. The optional values are ‘mean’, ‘sum’, and ‘none’. If ‘none’, do not perform reduction. Default:’mean’.
weight (Tensor, optional) – A rescaling weight applied to the loss of each batch element. If not None, it must can be broadcast to a tensor with shape of predict, data type must be float16 or float32. Default: None.
pos_weight (Tensor, optional) – A weight of positive examples. Must be a vector with length equal to the number of classes. If not None, it must can be broadcast to a tensor with shape of predict, data type must be float16 or float32. Default: None.
- Inputs:
predict (Tensor) - Input logits. The data type must be float16 or float32.
target (Tensor) - Ground truth label. Has the same data type and shape with predict.
- Outputs:
Scalar. If reduction is ‘none’, it’s a tensor with the same shape and type as input predict.
- Raises
TypeError – If data type of predict or target is neither float16 nor float32.
TypeError – If weight or pos_weight is Parameter.
TypeError – If data type of weight or pos_weight is neither float16 nor float32.
ValueError – If weight or pos_weight can not be broadcast to a tensor with shape of predict.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
- Supported Platforms:
Ascend
Examples
>>> predict = Tensor(np.array([[-0.8, 1.2, 0.7], [-0.1, -0.4, 0.7]]).astype(np.float32)) >>> target = Tensor(np.array([[0.3, 0.8, 1.2], [-0.6, 0.1, 2.2]]).astype(np.float32)) >>> loss = nn.BCEWithLogitsLoss() >>> output = loss(predict, target) >>> print(output) 0.3463612
-
class
tinyms.losses.CosineEmbeddingLoss(margin=0.0, reduction='mean')[source]¶ Computes the similarity between two tensors using cosine distance.
Given two tensors x1, x2, and a Tensor label y with values 1 or -1:
\[\begin{split}loss(x_1, x_2, y) = \begin{cases} 1-cos(x_1, x_2), & \text{if } y = 1\\ max(0, cos(x_1, x_2)-margin), & \text{if } y = -1\\ \end{cases}\end{split}\]- Parameters
- Inputs:
input_x1 (Tensor) - Input tensor.
input_x2 (Tensor) - Its shape and data type must be the same as input_x1’s shape and data type.
y (Tensor) - Contains value 1 or -1. Suppose the shape of input_x1 is \((x_1, x_2, x_3,..., x_R)\), then the shape of target must be \((x_1, x_3, x_4, ..., x_R)\).
- Outputs:
loss (Tensor) - If reduction is “none”, its shape is the same as y’s shape, otherwise a scalar value will be returned.
- Raises
TypeError – If margin is not a float.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
ValueError – If margin is not in range [-1, 1].
- Supported Platforms:
AscendGPU
Examples
>>> x1 = Tensor(np.array([[0.3, 0.8], [0.4, 0.3]]), mindspore.float32) >>> x2 = Tensor(np.array([[0.4, 1.2], [-0.4, -0.9]]), mindspore.float32) >>> y = Tensor(np.array([1, -1]), mindspore.int32) >>> cosine_embedding_loss = nn.CosineEmbeddingLoss() >>> output = cosine_embedding_loss(x1, x2, y) >>> print(output) 0.0003426075
-
class
tinyms.losses.SampledSoftmaxLoss(num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=True, seed=0, reduction='none')[source]¶ Computes the sampled softmax training loss.
- Parameters
num_sampled (int) – The number of classes to randomly sample per batch.
num_classes (int) – The number of possible classes.
num_true (int) – The number of target classes per training example.
sampled_values (Union[list, tuple]) – List or tuple of (sampled_candidates, true_expected_count, sampled_expected_count) returned by a *CandidateSampler function. Default to None, UniformCandidateSampler is applied.
remove_accidental_hits (bool) – Whether to remove “accidental hits” where a sampled class equals one of the target classes. Default is True.
seed (int) – Random seed for candidate sampling. Default: 0
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. If “none”, do not perform reduction. Default: “none”.
- Inputs:
weights (Tensor) - Tensor of shape (C, dim).
bias (Tensor) - Tensor of shape (C). The class biases.
labels (Tensor) - Tensor of shape (N, num_true), type int64, int32. The target classes.
inputs (Tensor) - Tensor of shape (N, dim). The forward activations of the input network.
- Outputs:
Tensor, a tensor of shape (N) with the per-example sampled softmax losses.
- Raises
TypeError – If sampled_values is not a list or tuple.
TypeError – If dtype of labels is neither int32 not int64.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
ValueError – If num_sampled or num_true is great than num_classes.
ValueError – If length of sampled_values is not equal to 3.
- Supported Platforms:
GPU
Examples
>>> mindspore.set_seed(1) >>> loss = nn.SampledSoftmaxLoss(num_sampled=4, num_classes=7, num_true=1) >>> weights = Tensor(np.random.randint(0, 9, [7, 10]), mindspore.float32) >>> biases = Tensor(np.random.randint(0, 9, [7]), mindspore.float32) >>> labels = Tensor([0, 1, 2]) >>> inputs = Tensor(np.random.randint(0, 9, [3, 10]), mindspore.float32) >>> output = loss(weights, biases, labels, inputs) >>> print(output) [4.6051701e+01 1.4000047e+01 6.1989022e-06]
-
class
tinyms.losses.DiceLoss(smooth=1e-05)[source]¶ The Dice coefficient is a set similarity loss. It is used to calculate the similarity between two samples. The value of the Dice coefficient is 1 when the segmentation result is the best and 0 when the segmentation result is the worst. The Dice coefficient indicates the ratio of the area between two objects to the total area. The function is shown as follows:
\[dice = 1 - \frac{2 * (pred \bigcap true)}{pred \bigcup true}\]- Parameters
smooth (float) – A term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-5.
- Inputs:
y_pred (Tensor) - Tensor of shape (N, …). The data type must be float16 or float32.
y (Tensor) - Tensor of shape (N, …). The data type must be float16 or float32.
- Outputs:
Tensor, a tensor of shape with the per-example sampled Dice losses.
- Raises
ValueError – If the dimensions are different.
TypeError – If the type of inputs are not Tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = nn.DiceLoss(smooth=1e-5) >>> y_pred = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]]), mstype.float32) >>> y = Tensor(np.array([[0, 1], [1, 0], [0, 1]]), mstype.float32) >>> output = loss(y_pred, y) >>> print(output) 0.38596618
-
class
tinyms.losses.MultiClassDiceLoss(weights=None, ignore_indiex=None, activation='softmax')[source]¶ When there are multiple classifications, label is transformed into multiple binary classifications by one hot. For each channel section in the channel, it can be regarded as a binary classification problem, so it can be obtained through the binary loss of each category, and then the average value.
- Parameters
weights (Union[Tensor, None]) – Tensor of shape [num_classes, dim]. The weight shape[0] should be equal to y shape[1].
activation (Union[str, Cell]) – Activate function applied to the output of the fully connected layer, eg. ‘ReLU’. Default: ‘softmax’. Choose from: [‘softmax’, ‘logsoftmax’, ‘relu’, ‘relu6’, ‘tanh’,’Sigmoid’]
- Inputs:
y_pred (Tensor) - Tensor of shape (N, C, …). The y_pred dimension should be greater than 1. The data type must be float16 or float32.
y (Tensor) - Tensor of shape (N, C, …). The y dimension should be greater than 1. The data type must be loat16 or float32.
- Outputs:
Tensor, a tensor of shape with the per-example sampled MultiClass Dice Losses.
- Raises
ValueError – If the shapes are different.
TypeError – If the type of inputs are not Tensor.
ValueError – If the dimension of y or y_pred is less than 2.
ValueError – If the weight shape[0] is not equal to y.shape[1].
ValueError – If weight is a tensor, but the dimension is not 2.
- Supported Platforms:
AscendGPU
Examples
>>> loss = nn.MultiClassDiceLoss(weights=None, ignore_indiex=None, activation="softmax") >>> y_pred = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]]), mstype.float32) >>> y = Tensor(np.array([[0, 1], [1, 0], [0, 1]]), mstype.float32) >>> output = loss(y_pred, y) >>> print(output) 0.3283009
-
class
tinyms.losses.RMSELoss[source]¶ RMSELoss creates a standard to measure the root mean square error between \(x\) and \(y\) element-wise, where \(x\) is the input and \(y\) is the target.
For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(M\) and \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:
\[\begin{split}loss = \begin{cases} \sqrt{\frac{1}{M}\sum_{m=1,n=1}^{M,N}{(x_m-y_n)^2}}, & \text {if M > N } \\\\ \sqrt{\frac{1}{N}\sum_{m=1,n=1}^{M,N}{(x_m-y_n)^2}}, &\text{if M < N } \end{cases}\end{split}\]- Inputs:
logits (Tensor) - Tensor of shape \((x_1, x_2, ..., x_M)\).
label (Tensor) - Tensor of shape \((y_1, y_2, ..., y_N)\).
- Outputs:
Tensor, weighted loss float tensor.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = nn.RMSELoss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) 0.57735026
-
class
tinyms.losses.MAELoss(reduction='mean')[source]¶ MAELoss creates a standard to measure the average absolute error between \(x\) and \(y\) element-wise, where \(x\) is the input and \(y\) is the target.
For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(M\) and \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:
\[\begin{split}MAE = \begin{cases} \sqrt{\frac{1}{M}\sum_{m=1,n=1}^{M,N}{|x_m-y_n|}}, & \text {if M > N } \\\\ \sqrt{\frac{1}{N}\sum_{m=1,n=1}^{M,N}{|x_m-y_n|}}, &\text{if M < N } \end{cases}\end{split}\]- Parameters
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.
- Inputs:
logits (Tensor) - Tensor of shape \((x_1, x_2, ..., x_M)\).
label (Tensor) - Tensor of shape \((y_1, y_2, ..., y_N)\).
- Outputs:
Tensor, weighted loss float tensor.
- Raises
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
- Supported Platforms:
AscendGPUCPU
Examples
>>> loss = nn.MAELoss() >>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32) >>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32) >>> output = loss(input_data, target_data) >>> print(output) 0.33333334
tinyms.optimizers¶
Optimizers module provides common optimizers for training, such as SGD, ADAM, Momentum. The optimizer is used to calculate and update the gradients.
-
class
tinyms.optimizers.Optimizer(learning_rate, parameters, weight_decay=0.0, loss_scale=1.0)[source]¶ Base class for all optimizers.
Note
This class defines the API to add Ops to train a model. Never use this class directly, but instead instantiate one of its subclasses.
Different parameter groups can set different learning_rate, weight_decay and grad_centralization.
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight_decay is positive. For most optimizer, when not separating parameters, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float.
parameters (Union[list[Parameter], list[dict]]) –
When the parameters is a list of Parameter which will be updated, the element in parameters must be class Parameter. When the parameters is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
weight_decay (Union[float, int]) – An int or a floating point value for the weight decay. It must be equal to or greater than 0. If the type of weight_decay input is int, it will be converted to float. Default: 0.0.
loss_scale (float) – A floating point value for the loss scale. It must be greater than 0. If the type of loss_scale input is int, it will be converted to float. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If loss_scale is not a float.
TypeError – If weight_decay is neither float nor int.
ValueError – If loss_scale is less than or equal to 0.
ValueError – If weight_decay is less than 0.
ValueError – If learning_rate is a Tensor, but the dimension of tensor is greater than 1.
- Supported Platforms:
AscendGPU
-
broadcast_params(optim_result)[source]¶ Apply Broadcast operations in the sequential order of parameter groups.
- Returns
bool, the status flag.
-
decay_weight(gradients)[source]¶ Weight decay.
An approach to reduce the overfitting of a deep learning neural network model.
-
get_lr()[source]¶ Get the learning rate of current step.
- Returns
float, the learning rate of current step.
-
gradients_centralization(gradients)[source]¶ Gradients centralization.
A method for optimizing convolutional layer parameters to impore the training speed of a deep learning neural network model.
-
scale_grad(gradients)[source]¶ Loss scale for mixed precision.
An approach of mixed precision training to improve the speed and energy efficiency of training deep neural network.
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
property
unique¶ The method is to see whether to make unique. The input type is bool. The method is read-only.
-
class
tinyms.optimizers.Momentum(params, learning_rate, momentum, weight_decay=0.0, loss_scale=1.0, use_nesterov=False)[source]¶ Implements the Momentum algorithm.
Refer to the paper on the importance of initialization and momentum in deep learning for more details.
\[v_{t} = v_{t-1} \ast u + gradients\]If use_nesterov is True:
\[p_{t} = p_{t-1} - (grad \ast lr + v_{t} \ast u \ast lr)\]If use_nesterov is False:
\[p_{t} = p_{t-1} - lr \ast v_{t}\]Here: where grad, lr, p, v and u denote the gradients, learning_rate, params, moments, and momentum respectively.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float.
momentum (float) – Hyperparameter of type float, means momentum for the moving average. It must be at least 0.0.
weight_decay (int, float) – Weight decay (L2 penalty). It must be equal to or greater than 0.0. Default: 0.0.
loss_scale (float) – A floating point value for the loss scale. It must be greater than 0.0. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.use_nesterov (bool) – Enable Nesterov momentum. Default: False.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
tuple[bool], all elements are True.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If loss_scale or momentum is not a float.
TypeError – If weight_decay is neither float nor int.
TypeError – If use_nesterov is not a bool.
ValueError – If loss_scale is less than or equal to 0.
ValueError – If weight_decay or momentum is less than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.Momentum(params=net.trainable_params(), learning_rate=0.1, momentum=0.9) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01, 'grad_centralization':True}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.Momentum(group_params, learning_rate=0.1, momentum=0.9, weight_decay=0.0) >>> # The conv_params's parameters will use a learning rate of default value 0.1 and a weight decay of 0.01 and >>> # grad centralization of True. >>> # The no_conv_params's parameters will use a learning rate of 0.01 and a weight decay of default value 0.0 >>> # and grad centralization of False.. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim, metrics=None)
-
class
tinyms.optimizers.LARS(optimizer, epsilon=1e-05, coefficient=0.001, use_clip=False, lars_filter=<function LARS.<lambda>>)[source]¶ Implements the LARS algorithm with LARSUpdate Operator.
LARS is an optimization algorithm employing a large batch optimization technique. Refer to paper LARGE BATCH TRAINING OF CONVOLUTIONAL NETWORKS.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ \lambda = \frac{\theta \text{ * } || \omega || }{|| g_{t} || \text{ + } \delta \text{ * } || \omega || } \\ \lambda = \begin{cases} \min(\frac{\lambda}{\alpha }, 1) & \text{ if } clip = True \\ \lambda & \text{ otherwise } \end{cases}\\ g_{t+1} = \lambda * (g_{t} + \delta * \omega) \end{array}\end{split}\]\(\theta\) represents coefficient, \(\omega\) represents parameters, \(g\) represents gradients, \(t\) represents updateing step, \(\delta\) represents weight_decay, \(\alpha\) represents learning_rate, \(clip\) represents use_clip.
- Parameters
optimizer (Optimizer) – MindSpore optimizer for which to wrap and modify gradients.
epsilon (float) – Term added to the denominator to improve numerical stability. Default: 1e-05.
coefficient (float) – Trust coefficient for calculating the local learning rate. Default: 0.001.
use_clip (bool) – Whether to use clip operation for calculating the local learning rate. Default: False.
lars_filter (Function) – A function to determine whether apply the LARS algorithm. Default: lambda x: ‘LayerNorm’ not in x.name and ‘bias’ not in x.name.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params in the optimizer, the shape is the as same as the params in the optimizer.
- Outputs:
Union[Tensor[bool], tuple[Parameter]], it depends on the output of optimizer.
- Supported Platforms:
Ascend
Examples
>>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> opt = nn.Momentum(net.trainable_params(), 0.1, 0.9) >>> opt_lars = nn.LARS(opt, epsilon=1e-08, coefficient=0.02) >>> model = Model(net, loss_fn=loss, optimizer=opt_lars, metrics=None)
-
class
tinyms.optimizers.Adam(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-08, use_locking=False, use_nesterov=False, weight_decay=0.0, loss_scale=1.0)[source]¶ Updates gradients by the Adaptive Moment Estimation (Adam) algorithm.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector moment1, \(v\) represents the 2nd moment vector moment2, \(g\) represents gradients, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents params, \(\epsilon\) represents eps.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters is supported.
The sparse strategy is applied while the SparseGatherV2 operator is used for forward network. The sparse feature is under continuous development. If the sparse strategy wants to be executed on the host, set the target to the CPU.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” is in the keys, the value of the corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” is in the keys, the value of the corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” is in the keys, the value must be the order of parameters and the order will be followed in the optimizer. There are no other keys in the dict and the parameters which in the ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use the dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Should be in range (0.0, 1.0). Default: 0.9.
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Should be in range (0.0, 1.0). Default: 0.999.
eps (float) – Term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-8.
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
loss_scale (float) – A floating point value for the loss scale. Should be greater than 0. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If beta1, beta2, eps or loss_scale is not a float.
TypeError – If weight_decay is neither float nor int.
TypeError – If use_locking or use_nesterov is not a bool.
ValueError – If loss_scale or eps is less than or equal to 0.
ValueError – If beta1, beta2 is not in range (0.0, 1.0).
ValueError – If weight_decay is less than 0.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.Adam(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01, 'grad_centralization':True}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.Adam(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01 and grad >>> # centralization of True. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0 and grad >>> # centralization of False. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
class
tinyms.optimizers.AdamWeightDecay(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-06, weight_decay=0.0)[source]¶ Implements the Adam algorithm to fix the weight decay.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” is in the keys, the value of the corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” is in the keys, the value of the corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” is in the keys, the value must be the order of parameters and the order will be followed in the optimizer. There are no other keys in the dict and the parameters which in the ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use the dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Default: 0.9. Should be in range (0.0, 1.0).
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Default: 0.999. Should be in range (0.0, 1.0).
eps (float) – Term added to the denominator to improve numerical stability. Default: 1e-6. Should be greater than 0.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
tuple[bool], all elements are True.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If beta1, beta2 or eps is not a float.
TypeError – If weight_decay is neither float nor int.
ValueError – If eps is less than or equal to 0.
ValueError – If beta1, beta2 is not in range (0.0, 1.0).
ValueError – If weight_decay is less than 0.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.AdamWeightDecay(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.AdamWeightDecay(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.LazyAdam(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-08, use_locking=False, use_nesterov=False, weight_decay=0.0, loss_scale=1.0)[source]¶ This optimizer will apply a lazy adam algorithm when gradient is sparse.
The original adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector moment1, \(v\) represents the 2nd moment vector moment2, \(g\) represents gradients, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents params, \(\epsilon\) represents eps.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters can be supported.
The sparse strategy is applied while the SparseGatherV2 operator being used for forward network. The sparse behavior, to be notice, is not equivalent to the original Adam algorithm, as only the current indices parames will be updated. The sparse feature is under continuous development. If the sparse strategy wants to be executed on the host, set the target to the CPU.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr” and “weight_decay” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Should be in range (0.0, 1.0). Default: 0.9.
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Should be in range (0.0, 1.0). Default: 0.999.
eps (float) – Term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-8.
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
weight_decay (Union[float, int]) – Weight decay (L2 penalty). Default: 0.0.
loss_scale (float) – A floating point value for the loss scale. Should be equal to or greater than 1. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If beta1, beta2, eps or loss_scale is not a float.
TypeError – If weight_decay is neither float nor int.
TypeError – If use_locking or use_nesterov is not a bool.
ValueError – If loss_scale or eps is less than or equal to 0.
ValueError – If beta1, beta2 is not in range (0.0, 1.0).
ValueError – If weight_decay is less than 0.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.LazyAdam(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01, 'grad_centralization':True}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.LazyAdam(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01 and grad >>> # centralization of True. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0 and grad >>> # centralization of False. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
class
tinyms.optimizers.AdamOffload(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-08, use_locking=False, use_nesterov=False, weight_decay=0.0, loss_scale=1.0)[source]¶ This optimizer will offload Adam optimizer to host CPU and keep parameters being updated on the device, to minimize the memory cost. Although that would bring about an increase of performance overhead, the optimizer could be used to run a larger model.
The Adam algorithm is proposed in Adam: A Method for Stochastic Optimization.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m = \beta_1 * m + (1 - \beta_1) * g \\ v = \beta_2 * v + (1 - \beta_2) * g * g \\ l = \alpha * \frac{\sqrt{1-\beta_2^t}}{1-\beta_1^t} \\ w = w - l * \frac{m}{\sqrt{v} + \epsilon} \end{array}\end{split}\]\(m\) represents the 1st moment vector moment1, \(v\) represents the 2nd moment vector moment2, \(g\) represents gradients, \(l\) represents scaling factor lr, \(\beta_1, \beta_2\) represent beta1 and beta2, \(t\) represents updating step while \(beta_1^t\) and \(beta_2^t\) represent beta1_power and beta2_power, \(\alpha\) represents learning_rate, \(w\) represents params, \(\epsilon\) represents eps.
Note
This optimizer only supports GRAPH_MODE currently.
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters is supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” is in the keys, the value of the corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” is in the keys, the value of the corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” is in the keys, the value must be the order of parameters and the order will be followed in the optimizer. There are no other keys in the dict and the parameters which in the ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use the dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Should be in range (0.0, 1.0). Default: 0.9.
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Should be in range (0.0, 1.0). Default: 0.999.
eps (float) – Term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-8.
use_locking (bool) – Whether to enable a lock to protect variable tensors from being updated. If true, updates of the var, m, and v tensors will be protected by a lock. If false, the result is unpredictable. Default: False.
use_nesterov (bool) – Whether to use Nesterov Accelerated Gradient (NAG) algorithm to update the gradients. If true, update the gradients using NAG. If false, update the gradients without using NAG. Default: False.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
loss_scale (float) – A floating point value for the loss scale. Should be greater than 0. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If beta1, beta2, eps or loss_scale is not a float.
TypeError – If weight_decay is neither float nor int.
TypeError – If use_locking or use_nesterov is not a bool.
ValueError – If loss_scale or eps is less than or equal to 0.
ValueError – If beta1, beta2 is not in range (0.0, 1.0).
ValueError – If weight_decay is less than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.AdamOffload(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.AdamOffload(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.Lamb(params, learning_rate, beta1=0.9, beta2=0.999, eps=1e-06, weight_decay=0.0)[source]¶ Lamb Dynamic Learning Rate.
LAMB is an optimization algorithm employing a layerwise adaptive large batch optimization technique. Refer to the paper LARGE BATCH OPTIMIZATION FOR DEEP LEARNING: TRAINING BERT IN 76 MINUTES.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Default: 0.9. Should be in range (0.0, 1.0).
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Default: 0.999. Should be in range (0.0, 1.0).
eps (float) – Term added to the denominator to improve numerical stability. Default: 1e-6. Should be greater than 0.
weight_decay (float) – Weight decay (L2 penalty). Default: 0.0. Should be equal to or greater than 0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
tuple[bool], all elements are True.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If beta1, beta2 or eps is not a float.
TypeError – If weight_decay is neither float nor int.
ValueError – If eps is less than or equal to 0.
ValueError – If beta1, beta2 is not in range (0.0, 1.0).
ValueError – If weight_decay is less than 0.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.Lamb(params=net.trainable_params(), learning_rate=0.1) >>> >>> #2) Use parameter groups and set different values >>> poly_decay_lr = learning_rate_schedule.PolynomialDecayLR(learning_rate=0.1, end_learning_rate=0.01, ... decay_steps=4, power = 0.5) >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01, 'grad_centralization':True}, ... {'params': no_conv_params, 'lr': poly_decay_lr}, ... {'order_params': net.trainable_params(0.01)}] >>> optim = nn.Lamb(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01 and grad >>> # centralization of True. >>> # The no_conv_params's parameters will use dynamic learning rate of poly decay learning rate and default >>> # weight decay of 0.0 and grad centralization of False. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.SGD(params, learning_rate=0.1, momentum=0.0, dampening=0.0, weight_decay=0.0, nesterov=False, loss_scale=1.0)[source]¶ Implements stochastic gradient descent. Momentum is optional.
Introduction to SGD can be found at https://en.wikipedia.org/wiki/Stochastic_gradient_descent. Nesterov momentum is based on the formula from paper On the importance of initialization and momentum in deep learning.
\[v_{t+1} = u \ast v_{t} + gradient \ast (1-dampening)\]If nesterov is True:
\[p_{t+1} = p_{t} - lr \ast (gradient + u \ast v_{t+1})\]If nesterov is False:
\[p_{t+1} = p_{t} - lr \ast v_{t+1}\]To be noticed, for the first step, v_{t+1} = gradient
Here : where p, v and u denote the parameters, accum, and momentum respectively.
Note
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 0.1.
momentum (float) – A floating point value the momentum. must be at least 0.0. Default: 0.0.
dampening (float) – A floating point value of dampening for momentum. must be at least 0.0. Default: 0.0.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
nesterov (bool) – Enables the Nesterov momentum. If use nesterov, momentum must be positive, and dampening must equal to 0.0. Default: False.
loss_scale (float) – A floating point value for the loss scale, which must be larger than 0.0. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Raises
ValueError – If the momentum, dampening or weight_decay value is less than 0.0.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.SGD(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params,'grad_centralization':True}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.SGD(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 default weight decay of 0.0 and grad >>> # centralization of True. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0 and grad >>> # centralization of False. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.FTRL(params, initial_accum=0.1, learning_rate=0.001, lr_power=-0.5, l1=0.0, l2=0.0, use_locking=False, loss_scale=1.0, weight_decay=0.0)[source]¶ Implements the FTRL algorithm with ApplyFtrl Operator.
FTRL is an online convex optimization algorithm that adaptively chooses its regularization function based on the loss functions. Refer to paper Adaptive Bound Optimization for Online Convex Optimization. Refer to paper Ad Click Prediction: a View from the Trenches for engineering document.
The updating formulas are as follows,
\[\begin{split}\begin{array}{ll} \\ m_{t+1} = m_{t} + g^2 \\ u_{t+1} = u_{t} + g - \frac{m_{t+1}^\text{-p} - m_{t}^\text{-p}}{\alpha } * \omega_{t} \\ \omega_{t+1} = \begin{cases} \frac{(sign(u_{t+1}) * l1 - u_{t+1})}{\frac{m_{t+1}^\text{-p}}{\alpha } + 2 * l2 } & \text{ if } |u_{t+1}| > l1 \\ 0.0 & \text{ otherwise } \end{cases}\\ \end{array}\end{split}\]\(m\) represents accum, \(g\) represents grads, \(t\) represents updating step, \(u\) represents linear, \(p\) represents lr_power, \(\alpha\) represents learning_rate, \(\omega\) represents params.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on all of the parameters.
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters can be supported.
The sparse strategy is applied while the SparseGatherV2 operator being used for forward network. The sparse feature is under continuous development. If the sparse strategy wants to be executed on the host, set the target to the CPU.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Using different learning rate by separating parameters is currently not supported.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
initial_accum (float) – The starting value for accumulators, must be zero or positive values. Default: 0.1.
learning_rate (float) – The learning rate value, must be zero or positive, dynamic learning rate is currently not supported. Default: 0.001.
lr_power (float) – Learning rate power controls how the learning rate decreases during training, must be less than or equal to zero. Use fixed learning rate if lr_power is zero. Default: -0.5.
l1 (float) – l1 regularization strength, must be greater than or equal to zero. Default: 0.0.
l2 (float) – l2 regularization strength, must be greater than or equal to zero. Default: 0.0.
use_locking (bool) – If true, use locks for updating operation. Default: False.
loss_scale (float) – Value for the loss scale. It must be greater than 0.0. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.weight_decay (Union[float, int]) – Weight decay value to multiply weight, must be zero or positive value. Default: 0.0.
- Inputs:
grads (tuple[Tensor]) - The gradients of params in the optimizer, the shape is the same as the params in optimizer.
- Outputs:
tuple[Parameter], the updated parameters, the shape is the same as params.
- Raises
TypeError – If initial_accum, learning_rate, lr_power, l1, l2 or loss_scale is not a float.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If weight_decay is neither float nor int.
TypeError – If use_nesterov is not a bool.
ValueError – If lr_power is greater than 0.
ValueError – If loss_scale is less than or equal to 0.
ValueError – If initial_accum, l1 or l2 is less than 0.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.FTRL(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01, 'grad_centralization':True}, ... {'params': no_conv_params}, ... {'order_params': net.trainable_params()}] >>> optim = nn.FTRL(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01 and grad >>> # centralization of True. >>> # The no_conv_params's parameters will use default weight decay of 0.0 and grad centralization of False. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
class
tinyms.optimizers.RMSProp(params, learning_rate=0.1, decay=0.9, momentum=0.0, epsilon=1e-10, use_locking=False, centered=False, loss_scale=1.0, weight_decay=0.0)[source]¶ Implements Root Mean Squared Propagation (RMSProp) algorithm.
Update params according to the RMSProp algorithm.
The equation is as follows:
\[s_{t} = \rho s_{t-1} + (1 - \rho)(\nabla Q_{i}(w))^2\]\[m_{t} = \beta m_{t-1} + \frac{\eta} {\sqrt{s_{t} + \epsilon}} \nabla Q_{i}(w)\]\[w = w - m_{t}\]The first equation calculates moving average of the squared gradient for each weight. Then dividing the gradient by \(\sqrt{ms_{t} + \epsilon}\).
if centered is True:
\[g_{t} = \rho g_{t-1} + (1 - \rho)\nabla Q_{i}(w)\]\[s_{t} = \rho s_{t-1} + (1 - \rho)(\nabla Q_{i}(w))^2\]\[m_{t} = \beta m_{t-1} + \frac{\eta} {\sqrt{s_{t} - g_{t}^2 + \epsilon}} \nabla Q_{i}(w)\]\[w = w - m_{t}\]where \(w\) represents params, which will be updated. \(g_{t}\) is mean gradients, \(g_{t-1}\) is the last moment of \(g_{t}\). \(s_{t}\) is the mean square gradients, \(s_{t-1}\) is the last moment of \(s_{t}\), \(m_{t}\) is moment, the delta of w, \(m_{t-1}\) is the last moment of \(m_{t}\). \(\rho\) represents decay. \(\beta\) is the momentum term, represents momentum. \(\epsilon\) is a smoothing term to avoid division by zero, represents epsilon. \(\eta\) is learning rate, represents learning_rate. \(\nabla Q_{i}(w)\) is gradients, represents gradients.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 0.1.
decay (float) – Decay rate. Should be equal to or greater than 0. Default: 0.9.
momentum (float) – Hyperparameter of type float, means momentum for the moving average. Should be equal to or greater than 0. Default: 0.0.
epsilon (float) – Term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-10.
use_locking (bool) – Whether to enable a lock to protect the variable and accumlation tensors from being updated. Default: False.
centered (bool) – If true, gradients are normalized by the estimated variance of the gradient. Default: False.
loss_scale (float) – A floating point value for the loss scale. Should be greater than 0. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.weight_decay (Union[float, int]) – Weight decay (L2 penalty). Should be equal to or greater than 0. Default: 0.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
Tensor[bool], the value is True.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If decay, momentum, epsilon or loss_scale is not a float.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If weight_decay is neither float nor int.
TypeError – If use_locking or centered is not a bool.
ValueError – If epsilon is less than or equal to 0.
ValueError – If decay or momentum is less than 0.
- Supported Platforms:
AscendGPUCPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.RMSProp(params=net.trainable_params(), learning_rate=0.1) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01, 'grad_centralization':True}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.RMSProp(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01 and grad >>> # centralization of True. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0 and grad >>> # centralization of False. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.ProximalAdagrad(params, accum=0.1, learning_rate=0.001, l1=0.0, l2=0.0, use_locking=False, loss_scale=1.0, weight_decay=0.0)[source]¶ Implements the ProximalAdagrad algorithm with ApplyProximalAdagrad Operator.
ProximalAdagrad is an online Learning and Stochastic Optimization. Refer to paper Efficient Learning using Forward-Backward Splitting.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters can be supported.
The sparse strategy is applied while the SparseGatherV2 operator being used for forward network. The sparse feature is under continuous development. If the sparse strategy wants to be executed on the host, set the target to the CPU.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
accum (float) – The starting value for accumulators, must be zero or positive values. Default: 0.1.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 0.001.
l1 (float) – l1 regularization strength, must be greater than or equal to zero. Default: 0.0.
l2 (float) – l2 regularization strength, must be greater than or equal to zero. Default: 0.0.
use_locking (bool) – If true, use locks for updating operation. Default: False.
loss_scale (float) – Value for the loss scale. It must be greater than 0.0. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.weight_decay (Union[float, int]) – Weight decay value to multiply weight, must be zero or positive value. Default: 0.0.
- Inputs:
grads (tuple[Tensor]) - The gradients of params in the optimizer, the shape is the same as the params in optimizer.
- Outputs:
Tensor[bool], the value is True.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If accum, l1, l2 or loss_scale is not a float.
TypeError – If weight_decay is neither float nor int.
ValueError – If loss_scale is less than or equal to 0.
ValueError – If accum, l1, l2 or weight_decay is less than 0.
- Supported Platforms:
Ascend
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.ProximalAdagrad(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01, 'grad_centralization':True}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.ProximalAdagrad(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01 and grad >>> # centralization of True. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0 and grad >>> # centralization of False. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
property
target¶ The method is used to determine whether the parameter is updated on host or device. The input type is str and can only be ‘CPU’, ‘Ascend’ or ‘GPU’.
-
class
tinyms.optimizers.Adagrad(params, accum=0.1, learning_rate=0.001, update_slots=True, loss_scale=1.0, weight_decay=0.0)[source]¶ Implements the Adagrad algorithm with ApplyAdagrad Operator.
Adagrad is an online Learning and Stochastic Optimization. Refer to paper Efficient Learning using Forward-Backward Splitting.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
When separating parameter groups, if you want to centralize the gradient, set grad_centralization to True, but the gradient centralization can only be applied to the parameters of the convolution layer. If the parameters of the non convolution layer are set to True, an error will be reported.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” in the keys, the value of corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” in the keys, the value of corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” in the keys, the value must be the order of parameters and the order will be followed in optimizer. There are no other keys in the dict and the parameters which in the value of ‘order_params’ must be in one of group parameters.
grad_centralization: Optional. The data type of “grad_centralization” is Bool. If “grad_centralization” is in the keys, the set value will be used. If not, the grad_centralization is False by default. This parameter only works on the convolution layer.
accum (float) – The starting value for accumulators, must be zero or positive values. Default: 0.1.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 0.001.
update_slots (bool) – If true, update accumulation. Default: True.
loss_scale (float) – Value for the loss scale. It must be greater than 0.0. In general, use the default value. Only when FixedLossScaleManager is used for training and the drop_overflow_update in FixedLossScaleManager is set to False, then this value needs to be the same as the loss_scale in FixedLossScaleManager. Refer to class
mindspore.FixedLossScaleManagerfor more details. Default: 1.0.weight_decay (Union[float, int]) – Weight decay value to multiply weight, must be zero or positive value. Default: 0.0.
- Inputs:
grads (tuple[Tensor]) - The gradients of params in the optimizer, the shape is the same as the params in optimizer.
- Outputs:
Tensor[bool], the value is True.
- Raises
TypeError – If learning_rate is not one of int, float, Tensor, Iterable, LearningRateSchedule.
TypeError – If element of parameters is neither Parameter nor dict.
TypeError – If accum or loss_scale is not a float.
TypeError – If update_slots is not a bool.
TypeError – If weight_decay is neither float nor int.
ValueError – If loss_scale is less than or equal to 0.
ValueError – If accum or weight_decay is less than 0.
- Supported Platforms:
AscendCPUGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = nn.Adagrad(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01, 'grad_centralization':True}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = nn.Adagrad(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01 and grad >>> # centralization of True. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0 and grad >>> # centralization of False. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> model = Model(net, loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.AdamWeightDecayForBert(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-06, weight_decay=0.0)[source]¶ Implements the Adam algorithm to fix the weight decay.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” is in the keys, the value of the corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” is in the keys, the value of the corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” is in the keys, the value must be the order of parameters and the order will be followed in the optimizer. There are no other keys in the dict and the parameters which in the ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use the dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Default: 0.9. Should be in range (0.0, 1.0).
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Default: 0.999. Should be in range (0.0, 1.0).
eps (float) – Term added to the denominator to improve numerical stability. Default: 1e-6. Should be greater than 0.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
overflow (tuple[Tensor]) - The overflow flag in dynamiclossscale.
- Outputs:
tuple[bool], all elements are True.
- Supported Platforms:
AscendGPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = AdamWeightDecay(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = AdamWeightDecay(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = layers.SoftmaxCrossEntropyWithLogits() >>> model = Model(net) >>> model.compile(loss_fn=loss, optimizer=optim)
-
class
tinyms.optimizers.AdamWeightDecayOp(params, learning_rate=0.001, beta1=0.9, beta2=0.999, eps=1e-06, weight_decay=0.0)[source]¶ Implements the Adam algorithm to fix the weight decay. It is a complete operator, not a combination of other ops.
Note
When separating parameter groups, the weight decay in each group will be applied on the parameters if the weight decay is positive. When not separating parameter groups, the weight_decay in the API will be applied on the parameters without ‘beta’ or ‘gamma’ in their names if weight_decay is positive.
To improve parameter groups performance, the customized order of parameters can be supported.
- Parameters
params (Union[list[Parameter], list[dict]]) –
When the params is a list of Parameter which will be updated, the element in params must be class Parameter. When the params is a list of dict, the “params”, “lr”, “weight_decay” and “order_params” are the keys can be parsed.
params: Required. The value must be a list of Parameter.
lr: Optional. If “lr” is in the keys, the value of the corresponding learning rate will be used. If not, the learning_rate in the API will be used.
weight_decay: Optional. If “weight_decay” is in the keys, the value of the corresponding weight decay will be used. If not, the weight_decay in the API will be used.
order_params: Optional. If “order_params” is in the keys, the value must be the order of parameters and the order will be followed in the optimizer. There are no other keys in the dict and the parameters which in the ‘order_params’ must be in one of group parameters.
learning_rate (Union[float, Tensor, Iterable, LearningRateSchedule]) – A value or a graph for the learning rate. When the learning_rate is an Iterable or a Tensor in a 1D dimension, use the dynamic learning rate, then the i-th step will take the i-th value as the learning rate. When the learning_rate is LearningRateSchedule, use dynamic learning rate, the i-th learning rate will be calculated during the process of training according to the formula of LearningRateSchedule. When the learning_rate is a float or a Tensor in a zero dimension, use fixed learning rate. Other cases are not supported. The float learning rate must be equal to or greater than 0. If the type of learning_rate is int, it will be converted to float. Default: 1e-3.
beta1 (float) – The exponential decay rate for the 1st moment estimations. Default: 0.9. Should be in range (0.0, 1.0).
beta2 (float) – The exponential decay rate for the 2nd moment estimations. Default: 0.999. Should be in range (0.0, 1.0).
eps (float) – Term added to the denominator to improve numerical stability. Default: 1e-6. Should be greater than 0.
weight_decay (float) – Weight decay (L2 penalty). It must be equal to or greater than 0. Default: 0.0.
- Inputs:
gradients (tuple[Tensor]) - The gradients of params, the shape is the same as params.
- Outputs:
tuple[bool], all elements are True.
- Supported Platforms:
GPU
Examples
>>> net = Net() >>> #1) All parameters use the same learning rate and weight decay >>> optim = AdamWeightDecayOp(params=net.trainable_params()) >>> >>> #2) Use parameter groups and set different values >>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params())) >>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params())) >>> group_params = [{'params': conv_params, 'weight_decay': 0.01}, ... {'params': no_conv_params, 'lr': 0.01}, ... {'order_params': net.trainable_params()}] >>> optim = AdamWeightDecayOp(group_params, learning_rate=0.1, weight_decay=0.0) >>> # The conv_params's parameters will use default learning rate of 0.1 and weight decay of 0.01. >>> # The no_conv_params's parameters will use learning rate of 0.01 and default weight decay of 0.0. >>> # The final parameters order in which the optimizer will be followed is the value of 'order_params'. >>> >>> loss = layers.SoftmaxCrossEntropyWithLogits() >>> model = Model(net) >>> model.compile(loss_fn=loss, optimizer=optim)
tinyms.callbacks¶
Callback related classes and functions in model training phase.
-
class
tinyms.callbacks.LossTimeMonitor(lr_init=None)[source]¶ Monitor loss and time.
- Parameters
lr_init (numpy.ndarray) – Train learning rate. Default: None.
- Returns
None
Examples
>>> from tinyms import Tensor >>> from tinyms.callbacks import LossTimeMonitor >>> >>> LossTimeMonitor(lr_init=Tensor([0.05] * 100).asnumpy())
-
class
tinyms.callbacks.BertLossCallBack(dataset_size=1)[source]¶ Monitor the loss in training. If the loss in NAN or INF terminating training.
- Parameters
dataset_size (int) – Print loss every times. Default: 1.
- Returns
None
Examples
>>> from tinyms.callbacks import BertLossCallBack >>> >>> BertLossCallBack(dataset_size=1)
-
class
tinyms.callbacks.Callback[source]¶ Abstract base class used to build a callback class. Callbacks are context managers which will be entered and exited when passing into the Model. You can use this mechanism to initialize and release resources automatically.
Callback function will execute some operations in the current step or epoch.
Examples
>>> class Print_info(Callback): >>> def step_end(self, run_context): >>> cb_params = run_context.original_args() >>> print(cb_params.cur_epoch_num) >>> print(cb_params.cur_step_num) >>> >>> print_cb = Print_info() >>> model.train(epoch, dataset, callbacks=print_cb)
-
begin(run_context)[source]¶ Called once before the network executing.
- Parameters
run_context (RunContext) – Include some information of the model.
-
end(run_context)[source]¶ Called once after network training.
- Parameters
run_context (RunContext) – Include some information of the model.
-
epoch_begin(run_context)[source]¶ Called before each epoch beginning.
- Parameters
run_context (RunContext) – Include some information of the model.
-
epoch_end(run_context)[source]¶ Called after each epoch finished.
- Parameters
run_context (RunContext) – Include some information of the model.
-
step_begin(run_context)[source]¶ Called before each epoch beginning.
- Parameters
run_context (RunContext) – Include some information of the model.
-
step_end(run_context)[source]¶ Called after each step finished.
- Parameters
run_context (RunContext) – Include some information of the model.
-
-
class
tinyms.callbacks.LossMonitor(per_print_times=1)[source]¶ Monitor the loss in training.
If the loss is NAN or INF, it will terminate training.
Note
If per_print_times is 0, do not print loss.
- Parameters
per_print_times (int) – Print the loss each every time. Default: 1.
- Raises
ValueError – If print_step is not an integer or less than zero.
-
class
tinyms.callbacks.TimeMonitor(data_size=None)[source]¶ Monitor the time in training.
- Parameters
data_size (int) – Dataset size. Default: None.
-
class
tinyms.callbacks.ModelCheckpoint(prefix='CKP', directory=None, config=None)[source]¶ The checkpoint callback class.
It is called to combine with train process and save the model and network parameters after training.
Note
In the distributed training scenario, please specify different directories for each training process to save the checkpoint file. Otherwise, the training may fail.
- Parameters
prefix (str) – The prefix name of checkpoint files. Default: “CKP”.
directory (str) – The path of the folder which will be saved in the checkpoint file. Default: None.
config (CheckpointConfig) – Checkpoint strategy configuration. Default: None.
- Raises
ValueError – If the prefix is invalid.
TypeError – If the config is not CheckpointConfig type.
-
end(run_context)[source]¶ Save the last checkpoint after training finished.
- Parameters
run_context (RunContext) – Context of the train running.
-
property
latest_ckpt_file_name¶ Return the latest checkpoint path and file name.
-
step_end(run_context)[source]¶ Save the checkpoint at the end of step.
- Parameters
run_context (RunContext) – Context of the train running.
-
class
tinyms.callbacks.SummaryCollector(summary_dir, collect_freq=10, collect_specified_data=None, keep_default_action=True, custom_lineage_data=None, collect_tensor_freq=None, max_file_size=None, export_options=None)[source]¶ SummaryCollector can help you to collect some common information.
It can help you to collect loss, learning late, computational graph and so on. SummaryCollector also enables the summary operator to collect data to summary files.
Note
Multiple SummaryCollector instances in callback list are not allowed.
Not all information is collected at the training phase or at the eval phase.
SummaryCollector always record the data collected by the summary operator.
SummaryCollector only supports Linux systems.
- Parameters
summary_dir (str) – The collected data will be persisted to this directory. If the directory does not exist, it will be created automatically.
collect_freq (int) – Set the frequency of data collection, it should be greater then zero, and the unit is step. If a frequency is set, we will collect data when (current steps % freq) equals to 0, and the first step will be collected at any time. It is important to note that if the data sink mode is used, the unit will become the epoch. It is not recommended to collect data too frequently, which can affect performance. Default: 10.
collect_specified_data (Union[None, dict]) –
Perform custom operations on the collected data. By default, if set to None, all data is collected as the default behavior. You can customize the collected data with a dictionary. For example, you can set {‘collect_metric’: False} to control not collecting metrics. The data that supports control is shown below. Default: None.
collect_metric (bool): Whether to collect training metrics, currently only the loss is collected. The first output will be treated as the loss and it will be averaged. Optional: True/False. Default: True.
collect_graph (bool): Whether to collect the computational graph. Currently, only training computational graph is collected. Optional: True/False. Default: True.
collect_train_lineage (bool): Whether to collect lineage data for the training phase, this field will be displayed on the lineage page of Mindinsight. Optional: True/False. Default: True.
collect_eval_lineage (bool): Whether to collect lineage data for the evaluation phase, this field will be displayed on the lineage page of Mindinsight. Optional: True/False. Default: True.
collect_input_data (bool): Whether to collect dataset for each training. Currently only image data is supported. If there are multiple columns of data in the dataset, the first column should be image data. Optional: True/False. Default: True.
collect_dataset_graph (bool): Whether to collect dataset graph for the training phase. Optional: True/False. Default: True.
histogram_regular (Union[str, None]): Collect weight and bias for parameter distribution page and displayed in MindInsight. This field allows regular strings to control which parameters to collect. It is not recommended to collect too many parameters at once, as it can affect performance. Note that if you collect too many parameters and run out of memory, the training will fail. Default: None, it means only the first five parameters are collected.
keep_default_action (bool) – This field affects the collection behavior of the ‘collect_specified_data’ field. True: it means that after specified data is set, non-specified data is collected as the default behavior. False: it means that after specified data is set, only the specified data is collected, and the others are not collected. Optional: True/False, Default: True.
custom_lineage_data (Union[dict, None]) – Allows you to customize the data and present it on the MingInsight lineage page. In the custom data, the type of the key supports str, and the type of value supports str, int and float. Default: None, it means there is no custom data.
collect_tensor_freq (Optional[int]) – The same semantics as the collect_freq, but controls TensorSummary only. Because TensorSummary data is too large to be compared with other summary data, this parameter is used to reduce its collection. By default, The maximum number of steps for collecting TensorSummary data is 20, but it will not exceed the number of steps for collecting other summary data. For example, given collect_freq=10, when the total steps is 600, TensorSummary will be collected 20 steps, while other summary data 61 steps, but when the total steps is 20, both TensorSummary and other summary will be collected 3 steps. Also note that when in parallel mode, the total steps will be split evenly, which will affect the number of steps TensorSummary will be collected. Default: None, which means to follow the behavior as described above.
max_file_size (Optional[int]) – The maximum size in bytes of each file that can be written to the disk. For example, to write not larger than 4GB, specify max_file_size=4*1024**3. Default: None, which means no limit.
export_options (Union[None, dict]) –
Perform custom operations on the export data. Note that the size of export files is not limited by the max_file_size. You can customize the export data with a dictionary. For example, you can set {‘tensor_format’: ‘npy’} to export tensor as npy file. The data that supports control is shown below. Default: None, it means that the data is not exported.
tensor_format (Union[str, None]): Customize the export tensor format. Supports [“npy”, None]. Default: None, it means that the tensor is not exported.
npy: export tensor as npy file.
- Raises
ValueError – If the parameter value is not expected.
TypeError – If the parameter type is not expected.
RuntimeError – If an error occurs during data collection.
Examples
>>> import mindspore.nn as nn >>> from mindspore import context >>> from mindspore.train.callback import SummaryCollector >>> from mindspore.train import Model >>> from mindspore.nn.metrics import Accuracy >>> >>> if __name__ == '__main__': ... # If the device_target is GPU, set the device_target to "GPU" ... context.set_context(mode=context.GRAPH_MODE, device_target="Ascend") ... mnist_dataset_dir = '/path/to/mnist_dataset_directory' ... # The detail of create_dataset method shown in model_zoo.official.cv.lenet.src.dataset.py ... ds_train = create_dataset(mnist_dataset_dir, 32) ... # The detail of LeNet5 shown in model_zoo.official.cv.lenet.src.lenet.py ... network = LeNet5(10) ... net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") ... net_opt = nn.Momentum(network.trainable_params(), 0.01, 0.9) ... model = Model(network, net_loss, net_opt, metrics={"Accuracy": Accuracy()}, amp_level="O2") ... ... # Simple usage: ... summary_collector = SummaryCollector(summary_dir='./summary_dir') ... model.train(1, ds_train, callbacks=[summary_collector], dataset_sink_mode=False) ... ... # Do not collect metric and collect the first layer parameter, others are collected by default ... specified={'collect_metric': False, 'histogram_regular': '^conv1.*'} ... summary_collector = SummaryCollector(summary_dir='./summary_dir', collect_specified_data=specified) ... model.train(1, ds_train, callbacks=[summary_collector], dataset_sink_mode=False)
-
class
tinyms.callbacks.CheckpointConfig(save_checkpoint_steps=1, save_checkpoint_seconds=0, keep_checkpoint_max=5, keep_checkpoint_per_n_minutes=0, integrated_save=True, async_save=False, saved_network=None)[source]¶ The configuration of model checkpoint.
Note
During the training process, if dataset is transmitted through the data channel, It is suggested to set ‘save_checkpoint_steps’ to an integer multiple of loop_size. Otherwise, the time to save the checkpoint may be biased.
- Parameters
save_checkpoint_steps (int) – Steps to save checkpoint. Default: 1.
save_checkpoint_seconds (int) – Seconds to save checkpoint. Can’t be used with save_checkpoint_steps at the same time. Default: 0.
keep_checkpoint_max (int) – Maximum number of checkpoint files can be saved. Default: 5.
keep_checkpoint_per_n_minutes (int) – Keep one checkpoint every n minutes. Can’t be used with keep_checkpoint_max at the same time. Default: 0.
integrated_save (bool) – Whether to perform integrated save function in automatic model parallel scene. Integrated save function is only supported in automatic parallel scene, not supported in manual parallel. Default: True.
async_save (bool) – Whether asynchronous execution saves the checkpoint to a file. Default: False.
saved_network (Cell) – Network to be saved in checkpoint file. If the saved_network has no relation with the network in training, the initial value of saved_network will be saved. Default: None.
- Raises
ValueError – If the input_param is None or 0.
Examples
>>> class LeNet5(nn.Cell): >>> def __init__(self, num_class=10, num_channel=1): >>> super(LeNet5, self).__init__() >>> self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid') >>> self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid') >>> self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02)) >>> self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02)) >>> self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02)) >>> self.relu = nn.ReLU() >>> self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) >>> self.flatten = nn.Flatten() >>> >>> def construct(self, x): >>> x = self.max_pool2d(self.relu(self.conv1(x))) >>> x = self.max_pool2d(self.relu(self.conv2(x))) >>> x = self.flatten(x) >>> x = self.relu(self.fc1(x)) >>> x = self.relu(self.fc2(x)) >>> x = self.fc3(x) >>> return x >>> >>> net = LeNet5() >>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean') >>> optim = nn.Momentum(net.trainable_params(), 0.01, 0.9) >>> model = Model(net, loss_fn=loss, optimizer=optim) >>> data_path = './MNIST_Data' >>> dataset = create_dataset(data_path) >>> config = CheckpointConfig(saved_network=net) >>> ckpoint_cb = ModelCheckpoint(prefix='LeNet5', directory='./checkpoint', config=config) >>> model.train(10, dataset, callbacks=ckpoint_cb)
-
property
async_save¶ Get the value of _async_save.
-
property
integrated_save¶ Get the value of _integrated_save.
-
property
keep_checkpoint_max¶ Get the value of _keep_checkpoint_max.
-
property
keep_checkpoint_per_n_minutes¶ Get the value of _keep_checkpoint_per_n_minutes.
-
property
save_checkpoint_seconds¶ Get the value of _save_checkpoint_seconds.
-
property
save_checkpoint_steps¶ Get the value of _save_checkpoint_steps.
-
property
saved_network¶ Get the value of _saved_network
-
class
tinyms.callbacks.RunContext(original_args)[source]¶ Provide information about the model.
Provide information about original request to model function. Callback objects can stop the loop by calling request_stop() of run_context.
- Parameters
original_args (dict) – Holding the related information of model.
-
get_stop_requested()[source]¶ Return whether a stop is requested or not.
- Returns
bool, if true, model.train() stops iterations.
-
class
tinyms.callbacks.LearningRateScheduler(learning_rate_function)[source]¶ Change the learning_rate during training.
Note
This class are not supported on CPU.
- Parameters
learning_rate_function (Function) – The function about how to change the learning rate during training.
Examples
>>> from _lr_scheduler_callback import LearningRateScheduler >>> import mindspore.nn as nn >>> from mindspore.train import Model ... >>> def learning_rate_function(lr, cur_step_num): ... if cur_step_num%1000 == 0: ... lr = lr*0.1 ... return lr ... >>> lr = 0.1 >>> momentum = 0.9 >>> net = Net() >>> loss = nn.SoftmaxCrossEntropyWithLogits() >>> optim = nn.Momentum(net.trainable_params(), learning_rate=lr, momentum=momentum) >>> model = Model(net, loss_fn=loss, optimizer=optim) ... >>> dataset = create_custom_dataset("custom_dataset_path") >>> model.train(1, dataset, callbacks=[LearningRateScheduler(learning_rate_function)], ... dataset_sink_mode=False)
tinyms.metrics¶
Metrics module provides functions to measure the performance of the machine learning models on the evaluation dataset. It’s used to choose the best model.
-
tinyms.metrics.names()[source]¶ Gets the names of the metric methods.
- Returns
List, the name list of metric methods.
-
tinyms.metrics.get_metric_fn(name, *args, **kwargs)[source]¶ Gets the metric method based on the input name.
- Parameters
name (str) – The name of metric method. Refer to the ‘__factory__’ object for the currently supported metrics.
args – Arguments for the metric function.
kwargs – Keyword arguments for the metric function.
- Returns
Metric object, class instance of the metric method.
Examples
>>> metric = nn.get_metric_fn('precision', eval_type='classification')
-
class
tinyms.metrics.Accuracy(eval_type='classification')[source]¶ Calculates the accuracy for classification and multilabel data.
The accuracy class creates two local variables, the correct number and the total number that are used to compute the frequency with which predictions matches labels. This frequency is ultimately returned as the accuracy: an idempotent operation that simply divides the correct number by the total number.
\[\text{accuracy} =\frac{\text{true_positive} + \text{true_negative}} {\text{true_positive} + \text{true_negative} + \text{false_positive} + \text{false_negative}}\]- Parameters
eval_type (str) – Metric to calculate the accuracy over a dataset, for classification (single-label), and multilabel (multilabel classification). Default: ‘classification’.
Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]]), mindspore.float32) >>> y = Tensor(np.array([1, 0, 1]), mindspore.float32) >>> metric = nn.Accuracy('classification') >>> metric.clear() >>> metric.update(x, y) >>> accuracy = metric.eval() >>> print(accuracy) 0.6666666666666666
-
eval()[source]¶ Computes the accuracy.
- Returns
Float, the computed result.
- Raises
RuntimeError – If the sample size is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result \(y_{pred}\) and \(y\).
- Parameters
inputs – Input y_pred and y. y_pred and y are a Tensor, a list or an array. For the ‘classification’ evaluation type, y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. Shape of y can be \((N, C)\) with values 0 and 1 if one-hot encoding is used or the shape is \((N,)\) with integer values if index of category is used. For ‘multilabel’ evaluation type, y_pred and y can only be one-hot encoding with values 0 or 1. Indices with 1 indicate the positive category. The shape of y_pred and y are both \((N, C)\).
- Raises
ValueError – If the number of the inputs is not 2.
-
class
tinyms.metrics.MAE[source]¶ Calculates the mean absolute error.
Creates a criterion that measures the mean absolute error (MAE) between each element in the input: \(x\) and the target: \(y\).
\[\text{MAE} = \frac{\sum_{i=1}^n \|y_i - x_i\|}{n}\]Here \(y_i\) is the prediction and \(x_i\) is the true value.
Note
The method update must be called with the form update(y_pred, y).
Examples
>>> x = Tensor(np.array([0.1, 0.2, 0.6, 0.9]), mindspore.float32) >>> y = Tensor(np.array([0.1, 0.25, 0.7, 0.9]), mindspore.float32) >>> error = nn.MAE() >>> error.clear() >>> error.update(x, y) >>> result = error.eval() >>> print(result) 0.037499990314245224
-
eval()[source]¶ Computes the mean absolute error.
- Returns
Float, the computed result.
- Raises
RuntimeError – If the number of the total samples is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result \(y_{pred}\) and \(y\).
- Parameters
inputs – Input y_pred and y for calculating mean absolute error where the shape of y_pred and y are both N-D and the shape are the same.
- Raises
ValueError – If the number of the input is not 2.
-
-
class
tinyms.metrics.MSE[source]¶ Measures the mean squared error.
Creates a criterion that measures the mean squared error (squared L2 norm) between each element in the input: \(x\) and the target: \(y\).
\[\text{MSE}(x,\ y) = \frac{\sum_{i=1}^n(y_i - x_i)^2}{n}\]where \(n\) is batch size.
Examples
>>> x = Tensor(np.array([0.1, 0.2, 0.6, 0.9]), mindspore.float32) >>> y = Tensor(np.array([0.1, 0.25, 0.5, 0.9]), mindspore.float32) >>> error = nn.MSE() >>> error.clear() >>> error.update(x, y) >>> result = error.eval()
-
eval()[source]¶ Compute the mean squared error.
- Returns
Float, the computed result.
- Raises
RuntimeError – If the number of samples is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result \(y_{pred}\) and \(y\).
- Parameters
inputs – Input y_pred and y for calculating mean square error where the shape of y_pred and y are both N-D and the shape are the same.
- Raises
ValueError – If the number of input is not 2.
-
-
class
tinyms.metrics.Metric[source]¶ Base class of metric.
Note
For examples of subclasses, please refer to the definition of class MAE, ‘Recall’ etc.
-
abstract
clear()[source]¶ An interface describes the behavior of clearing the internal evaluation result.
Note
All subclasses must override this interface.
-
abstract
-
class
tinyms.metrics.Precision(eval_type='classification')[source]¶ Calculates precision for classification and multilabel data.
The precision function creates two local variables, \(\text{true_positive}\) and \(\text{false_positive}\), that are used to compute the precision. This value is ultimately returned as the precision, an idempotent operation that simply divides \(\text{true_positive}\) by the sum of \(\text{true_positive}\) and \(\text{false_positive}\).
\[\text{precision} = \frac{\text{true_positive}}{\text{true_positive} + \text{false_positive}}\]Note
In the multi-label cases, the elements of \(y\) and \(y_{pred}\) must be 0 or 1.
- Parameters
eval_type (str) – Metric to calculate accuracy over a dataset, for classification or multilabel. Default: ‘classification’.
Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([1, 0, 1])) >>> metric = nn.Precision('classification') >>> metric.clear() >>> metric.update(x, y) >>> precision = metric.eval() >>> print(precision) [0.5 1. ]
-
eval(average=False)[source]¶ Computes the precision.
- Parameters
average (bool) – Specify whether calculate the average precision. Default value is False.
- Returns
Float, the computed result.
-
update(*inputs)[source]¶ Updates the internal evaluation result with y_pred and y.
- Parameters
inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray. For ‘classification’ evaluation type, y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. Shape of y can be \((N, C)\) with values 0 and 1 if one-hot encoding is used or the shape is \((N,)\) with integer values if index of category is used. For ‘multilabel’ evaluation type, y_pred and y can only be one-hot encoding with values 0 or 1. Indices with 1 indicate positive category. The shape of y_pred and y are both \((N, C)\).
- Raises
ValueError – If the number of input is not 2.
-
class
tinyms.metrics.HausdorffDistance(distance_metric='euclidean', percentile=None, directed=False, crop=True)[source]¶ Calculates the Hausdorff distance. Hausdorff distance is the maximum and minimum distance between two point sets. Given two feature sets A and B, the Hausdorff distance between two point sets A and B is defined as follows:
\[H(A, B) = \text{max}[h(A, B), h(B, A)] h(A, B) = \underset{a \in A}{\text{max}}\{\underset{b \in B}{\text{min}} \rVert a - b \rVert \} h(A, B) = \underset{b \in B}{\text{max}}\{\underset{a \in A}{\text{min}} \rVert b - a \rVert \}\]- Parameters
distance_metric (string) – The parameter of calculating Hausdorff distance supports three measurement methods, “euclidean”, “chessboard” or “taxicab”. Default: “euclidean”.
percentile (float) – Floating point numbers between 0 and 100. Specify the percentile parameter to get the percentile of the Hausdorff distance. Defaults: None.
directed (bool) – It can be divided into directional and non directional Hausdorff distance, and the default is non directional Hausdorff distance, specify the percentile parameter to get the percentile of the Hausdorff distance. Default: False.
crop (bool) – Crop input images and only keep the foregrounds. In order to maintain two inputs’ shapes, here the bounding box is achieved by (y_pred | y) which represents the union set of two images. Default: True.
- Supported Platforms:
AscendGPUCPU
Examples
>>> x = Tensor(np.array([[3, 0, 1], [1, 3, 0], [1, 0, 2]])) >>> y = Tensor(np.array([[0, 2, 1], [1, 2, 1], [0, 0, 1]])) >>> metric = nn.HausdorffDistance() >>> metric.clear() >>> metric.update(x, y, 0) >>> mean_average_distance = metric.eval() >>> print(mean_average_distance) 1.4142135623730951
-
update(*inputs)[source]¶ Updates the internal evaluation result ‘y_pred’, ‘y’ and ‘label_idx’.
- Parameters
inputs –
- Input ‘y_pred’, ‘y’ and ‘label_idx’. ‘y_pred’ and ‘y’ are Tensor or numpy.ndarray. ‘y_pred’ is the
predicted binary image. ‘y’ is the actual binary image. ‘label_idx’, the data type of label_idx is int.
- Raises:
ValueError: If the number of the inputs is not 3.
-
class
tinyms.metrics.Recall(eval_type='classification')[source]¶ Calculates recall for classification and multilabel data.
The recall class creates two local variables, \(\text{true_positive}\) and \(\text{false_negative}\), that are used to compute the recall. This value is ultimately returned as the recall, an idempotent operation that simply divides \(\text{true_positive}\) by the sum of \(\text{true_positive}\) and \(\text{false_negative}\).
\[\text{recall} = \frac{\text{true_positive}}{\text{true_positive} + \text{false_negative}}\]Note
In the multi-label cases, the elements of \(y\) and \(y_{pred}\) must be 0 or 1.
- Parameters
eval_type (str) – Metric to calculate the recall over a dataset, for classification or multilabel. Default: ‘classification’.
Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([1, 0, 1])) >>> metric = nn.Recall('classification') >>> metric.clear() >>> metric.update(x, y) >>> recall = metric.eval() >>> print(recall) [1. 0.5]
-
eval(average=False)[source]¶ Computes the recall.
- Parameters
average (bool) – Specify whether calculate the average recall. Default value is False.
- Returns
Float, the computed result.
-
update(*inputs)[source]¶ Updates the internal evaluation result with y_pred and y.
- Parameters
inputs – Input y_pred and y. y_pred and y are a Tensor, a list or an array. For ‘classification’ evaluation type, y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. Shape of y can be \((N, C)\) with values 0 and 1 if one-hot encoding is used or the shape is \((N,)\) with integer values if index of category is used. For ‘multilabel’ evaluation type, y_pred and y can only be one-hot encoding with values 0 or 1. Indices with 1 indicate positive category. The shape of y_pred and y are both \((N, C)\).
- Raises
ValueError – If the number of input is not 2.
-
class
tinyms.metrics.Fbeta(beta)[source]¶ Calculates the fbeta score.
Fbeta score is a weighted mean of precison and recall.
\[F_\beta=\frac{(1+\beta^2) \cdot true\_positive} {(1+\beta^2) \cdot true\_positive +\beta^2 \cdot false\_negative + false\_positive}\]Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([1, 0, 1])) >>> metric = nn.Fbeta(1) >>> metric.clear() >>> metric.update(x, y) >>> fbeta = metric.eval() >>> print(fbeta) [0.66666667 0.66666667]
-
eval(average=False)[source]¶ Computes the fbeta.
- Parameters
average (bool) – Whether to calculate the average fbeta. Default value is False.
- Returns
Float, computed result.
-
update(*inputs)[source]¶ Updates the internal evaluation result y_pred and y.
- Parameters
inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray. y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. y contains values of integers. The shape is \((N, C)\) if one-hot encoding is used. Shape can also be \((N,)\) if category index is used.
-
-
class
tinyms.metrics.BleuScore(n_gram=4, smooth=False)[source]¶ Calculates BLEU score of machine translated text with one or more references.
- Parameters
Example
>>> candidate_corpus = [['i', 'have', 'a', 'pen', 'on', 'my', 'desk']] >>> reference_corpus = [[['i', 'have', 'a', 'pen', 'in', 'my', 'desk'], ... ['there', 'is', 'a', 'pen', 'on', 'the', 'desk']]] >>> metric = BleuScore() >>> metric.clear() >>> metric.update(candidate_corpus, reference_corpus) >>> bleu_score = metric.eval() >>> print(bleu_score) 0.5946035575013605
-
update(*inputs)[source]¶ Updates the internal evaluation result with candidate_corpus and reference_corpus.
- Parameters
inputs – Input candidate_corpus and reference_corpus. candidate_corpus and reference_corpus are a list. The candidate_corpus is an iterable of machine translated corpus. The reference_corpus is an iterable of iterables of reference corpus.
- Raises
ValueError – If the number of input is not 2.
-
class
tinyms.metrics.CosineSimilarity(similarity='cosine', reduction='none', zero_diagonal=True)[source]¶ Computes representation similarity
- Parameters
- Returns
A square matrix (input1, input1) with the similarity scores between all elements. If sum or mean are used, then returns (b, 1) with the reduced value for each row
Example
>>> test_data = np.array([[1, 3, 4, 7], [2, 4, 2, 5], [3, 1, 5, 8]]) >>> metric = CosineSimilarity() >>> metric.clear() >>> metric.update(test_data) >>> square_matrix = metric.eval() >>> print(square_matrix) [[0. 0.94025615 0.95162452] [0.94025615 0. 0.86146098] [0.95162452 0.86146098 0.]]
-
class
tinyms.metrics.OcclusionSensitivity(pad_val=0.0, margin=2, n_batch=128, b_box=None)[source]¶ This function is used to calculate the occlusion sensitivity of the model for a given image. Occlusion sensitivity refers to how the probability of a given prediction changes with the change of the occluded part of the image.
For a given result, the output probability is the probability of a region.
The higher the value in the output image, the greater the decline of certainty, indicating that the occluded area is more important in the decision-making process.
- Parameters
pad_val (float) – What values need to be entered in the image when a part of the image is occluded. Default: 0.0.
margin (Union[int, Sequence]) – Create a cuboid / cube around the voxel you want to occlude. Default: 2.
n_batch (int) – number of images in a batch before inference. Default: 128.
b_box (Sequence) – Bounding box on which to perform the analysis. The output image will also match in size. There should be a minimum and maximum for all dimensions except batch:
[min1, max1, min2, max2,...]. If no bounding box is supplied, this will be the same size as the input image. If a bounding box is used, the output image will be cropped to this size. Default: None.
Example
>>> class DenseNet(nn.Cell): ... def __init__(self): ... super(DenseNet, self).__init__() ... w = np.array([[0.1, 0.8, 0.1, 0.1],[1, 1, 1, 1]]).astype(np.float32) ... b = np.array([0.3, 0.6]).astype(np.float32) ... self.dense = nn.Dense(4, 2, weight_init=Tensor(w), bias_init=Tensor(b)) ... ... def construct(self, x): ... return self.dense(x) >>> >>> model = DenseNet() >>> test_data = np.array([[0.1, 0.2, 0.3, 0.4]]).astype(np.float32) >>> label = np.array(1).astype(np.int32) >>> metric = OcclusionSensitivity() >>> metric.clear() >>> metric.update(model, test_data, label) >>> score = metric.eval() >>> print(score) [0.29999995 0.6 1 0.9]
-
update(*inputs)[source]¶ Updates input, including model, y_pred and label.
- Inputs:
model (nn.Cell) - classification model to use for inference.
y_pred (Union[Tensor, list, np.ndarray]) - image to test. Should be tensor consisting of 1 batch, can be 2- or 3D.
label (Union[int, Tensor]) - classification label to check for changes (normally the true label, but doesn’t have to be
- Raises
ValueError – If the number of input is not 3.
-
class
tinyms.metrics.F1[source]¶ Calculates the F1 score. F1 is a special case of Fbeta when beta is 1. Refer to class
mindspore.nn.Fbetafor more details.\[F_1=\frac{2\cdot true\_positive}{2\cdot true\_positive + false\_negative + false\_positive}\]Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([1, 0, 1])) >>> metric = nn.F1() >>> metric.update(x, y) >>> result = metric.eval() >>> print(result) [0.66666667 0.66666667]
-
class
tinyms.metrics.Dice(smooth=1e-05)[source]¶ The Dice coefficient is a set similarity metric. It is used to calculate the similarity between two samples. The value of the Dice coefficient is 1 when the segmentation result is the best and 0 when the segmentation result is the worst. The Dice coefficient indicates the ratio of the area between two objects to the total area. The function is shown as follows:
\[dice = \frac{2 * (pred \bigcap true)}{pred \bigcup true}\]- Parameters
smooth (float) – A term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-5.
Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([[0, 1], [1, 0], [0, 1]])) >>> metric = Dice(smooth=1e-5) >>> metric.clear() >>> metric.update(x, y) >>> dice = metric.eval() >>> print(dice) 0.20467791371802546
-
eval()[source]¶ Computes the Dice.
- Returns
Float, the computed result.
- Raises
RuntimeError – If the total samples num is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result \(y_pred\) and \(y\).
- Parameters
inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray. y_pred is the predicted value, y is the true value. The shape of y_pred and y are both \((N, ...)\).
- Raises
ValueError – If the number of the inputs is not 2.
-
class
tinyms.metrics.ROC(class_num=None, pos_label=None)[source]¶ Calculates the ROC curve. It is suitable for solving binary classification and multi classification problems. In the case of multiclass, the values will be calculated based on a one-vs-the-rest approach.
- Parameters
class_num (int) – Integer with the number of classes. For the problem of binary classification, it is not necessary to provide this argument. Default: None.
pos_label (int) – Determine the integer of positive class. Default: None. For binary problems, it is translated to 1. For multiclass problems, this argument should not be set, as it is iteratively changed in the range [0,num_classes-1]. Default: None.
Examples
>>> # 1) binary classification example >>> x = Tensor(np.array([3, 1, 4, 2])) >>> y = Tensor(np.array([0, 1, 2, 3])) >>> metric = ROC(pos_label=2) >>> metric.clear() >>> metric.update(x, y) >>> fpr, tpr, thresholds = metric.eval() >>> print(fpr) [0. 0. 0.33333333 0.6666667 1.] >>> print(tpr) [0. 1. 1. 1. 1.] >>> print(thresholds) [5 4 3 2 1] >>> >>> # 2) multiclass classification example >>> x = Tensor(np.array([[0.28, 0.55, 0.15, 0.05], [0.10, 0.20, 0.05, 0.05], [0.20, 0.05, 0.15, 0.05], ... [0.05, 0.05, 0.05, 0.75]])) >>> y = Tensor(np.array([0, 1, 2, 3])) >>> metric = ROC(class_num=4) >>> metric.clear() >>> metric.update(x, y) >>> fpr, tpr, thresholds = metric.eval() >>> print(fpr) [array([0., 0., 0.33333333, 0.66666667, 1.]), array([0., 0.33333333, 0.33333333, 1.]), array([0., 0.33333333, 1.]), array([0., 0., 1.])] >>> print(tpr) [array([0., 1., 1., 1., 1.]), array([0., 0., 1., 1.]), array([0., 1., 1.]), array([0., 1., 1.])] print(thresholds) [array([1.28, 0.28, 0.2, 0.1, 0.05]), array([1.55, 0.55, 0.2, 0.05]), array([1.15, 0.15, 0.05]), array([1.75, 0.75, 0.05])]
-
eval()[source]¶ Computes the ROC curve.
- Returns
A tuple, composed of fpr, tpr, and thresholds.
fpr (np.array) - np.array with false positive rates. If multiclass, this is a list of such np.array, one for each class.
tps (np.array) - np.array with true positive rates. If multiclass, this is a list of such np.array, one for each class.
thresholds (np.array) - thresholds used for computing false- and true positive rates.
-
update(*inputs)[source]¶ Update state with predictions and targets.
- Parameters
inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray. In most cases (not strictly), y_pred is a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. y contains values of integers.
-
tinyms.metrics.auc(x, y, reorder=False)[source]¶ Computes the Area Under the Curve (AUC) using the trapezoidal rule. This is a general function, given points on a curve. For computing the area under the ROC-curve.
- Parameters
x (Union[np.array, list]) – From the ROC curve(fpr), np.array with false positive rates. If multiclass, this is a list of such np.array, one for each class. The shape \((N)\).
y (Union[np.array, list]) – From the ROC curve(tpr), np.array with true positive rates. If multiclass, this is a list of such np.array, one for each class. The shape \((N)\).
reorder (boolean) – If True, assume that the curve is ascending in the case of ties, as for an ROC curve. If the curve is non-ascending, the result will be wrong. Default: False.
- Returns
Compute result.
- Return type
area (float)
Examples
>>> y_pred = np.array([[3, 0, 1], [1, 3, 0], [1, 0, 2]]) >>> y = np.array([[0, 2, 1], [1, 2, 1], [0, 0, 1]]) >>> metric = nn.ROC(pos_label=2) >>> metric.clear() >>> metric.update(y_pred, y) >>> fpr, tpr, thre = metric.eval() >>> output = auc(fpr, tpr) >>> print(output) 0.5357142857142857
-
class
tinyms.metrics.TopKCategoricalAccuracy(k)[source]¶ Calculates the top-k categorical accuracy.
Note
The method update must receive input of the form \((y_{pred}, y)\). If some samples have the same accuracy, the first sample will be chosen.
- Parameters
k (int) – Specifies the top-k categorical accuracy to compute.
- Raises
TypeError – If k is not int.
ValueError – If k is less than 1.
Examples
>>> x = Tensor(np.array([[0.2, 0.5, 0.3, 0.6, 0.2], [0.1, 0.35, 0.5, 0.2, 0.], ... [0.9, 0.6, 0.2, 0.01, 0.3]]), mindspore.float32) >>> y = Tensor(np.array([2, 0, 1]), mindspore.float32) >>> topk = nn.TopKCategoricalAccuracy(3) >>> topk.clear() >>> topk.update(x, y) >>> output = topk.eval() >>> print(output) 0.6666666666666666
-
update(*inputs)[source]¶ Updates the internal evaluation result y_pred and y.
- Parameters
inputs – Input y_pred and y. y_pred and y are Tensor, list or numpy.ndarray. y_pred is in most cases (not strictly) a list of floating numbers in range \([0, 1]\) and the shape is \((N, C)\), where \(N\) is the number of cases and \(C\) is the number of categories. y contains values of integers. The shape is \((N, C)\) if one-hot encoding is used. Shape can also be \((N,)\) if category index is used.
-
class
tinyms.metrics.Top1CategoricalAccuracy[source]¶ Calculates the top-1 categorical accuracy. This class is a specialized class for TopKCategoricalAccuracy. Refer to class ‘TopKCategoricalAccuracy’ for more details.
Examples
>>> x = Tensor(np.array([[0.2, 0.5, 0.3, 0.6, 0.2], [0.1, 0.35, 0.5, 0.2, 0.], ... [0.9, 0.6, 0.2, 0.01, 0.3]]), mindspore.float32) >>> y = Tensor(np.array([2, 0, 1]), mindspore.float32) >>> topk = nn.Top1CategoricalAccuracy() >>> topk.clear() >>> topk.update(x, y) >>> output = topk.eval() >>> print(output) 0.0
-
class
tinyms.metrics.Top5CategoricalAccuracy[source]¶ Calculates the top-5 categorical accuracy. This class is a specialized class for TopKCategoricalAccuracy. Refer to class ‘TopKCategoricalAccuracy’ for more details.
Examples
>>> x = Tensor(np.array([[0.2, 0.5, 0.3, 0.6, 0.2], [0.1, 0.35, 0.5, 0.2, 0.], ... [0.9, 0.6, 0.2, 0.01, 0.3]]), mindspore.float32) >>> y = Tensor(np.array([2, 0, 1]), mindspore.float32) >>> topk = nn.Top5CategoricalAccuracy() >>> topk.clear() >>> topk.update(x, y) >>> output = topk.eval() >>> print(output) 1.0
-
class
tinyms.metrics.Loss[source]¶ Calculates the average of the loss. If method ‘update’ is called every \(n\) iterations, the result of evaluation will be:
\[loss = \frac{\sum_{k=1}^{n}loss_k}{n}\]Examples
>>> x = Tensor(np.array(0.2), mindspore.float32) >>> loss = nn.Loss() >>> loss.clear() >>> loss.update(x) >>> result = loss.eval()
-
eval()[source]¶ Calculates the average of the loss.
- Returns
Float, the average of the loss.
- Raises
RuntimeError – If the total number is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result.
- Parameters
inputs – Inputs contain only one element, the element is loss. The dimension of loss must be 0 or 1.
- Raises
ValueError – If the length of inputs is not 1.
ValueError – If the dimensions of loss is not 1.
-
-
class
tinyms.metrics.MeanSurfaceDistance(symmetric=False, distance_metric='euclidean')[source]¶ This function is used to compute the Average Surface Distance from y_pred to y under the default setting. Mean Surface Distance(MSD), the mean of the vector is taken. This tell us how much, on average, the surface varies between the segmentation and the GT.
- Parameters
distance_metric (string) – The parameter of calculating Hausdorff distance supports three measurement methods, “euclidean”, “chessboard” or “taxicab”. Default: “euclidean”.
symmetric (bool) – if calculate the symmetric average surface distance between y_pred and y. In addition, if sets
symmetric = True, the average symmetric surface distance between these two inputs will be returned. Defaults: False.
Examples
>>> x = Tensor(np.array([[3, 0, 1], [1, 3, 0], [1, 0, 2]])) >>> y = Tensor(np.array([[0, 2, 1], [1, 2, 1], [0, 0, 1]])) >>> metric = nn.MeanSurfaceDistance(symmetric=False, distance_metric="euclidean") >>> metric.clear() >>> metric.update(x, y, 0) >>> mean_average_distance = metric.eval() >>> print(mean_average_distance) 0.8047378541243649
-
update(*inputs)[source]¶ Updates the internal evaluation result ‘y_pred’, ‘y’ and ‘label_idx’.
- Parameters
inputs –
- Input ‘y_pred’, ‘y’ and ‘label_idx’. ‘y_pred’ and ‘y’ are Tensor or numpy.ndarray. ‘y_pred’ is the
predicted binary image. ‘y’ is the actual binary image. ‘label_idx’, the data type of label_idx is int.
- Raises:
ValueError: If the number of the inputs is not 3.
-
class
tinyms.metrics.RootMeanSquareDistance(symmetric=False, distance_metric='euclidean')[source]¶ This function is used to compute the Residual Mean Square Distance from y_pred to y under the default setting. Residual Mean Square Distance(RMS), the mean is taken from each of the points in the vector, these residuals are squared (to remove negative signs), summed, weighted by the mean and then the square-root is taken. Measured in mm.
- Parameters
distance_metric (string) – The parameter of calculating Hausdorff distance supports three measurement methods, “euclidean”, “chessboard” or “taxicab”. Default: “euclidean”.
symmetric (bool) – if calculate the symmetric average surface distance between y_pred and y. In addition, if sets
symmetric = True, the average symmetric surface distance between these two inputs will be returned. Defaults: False.
Examples
>>> x = Tensor(np.array([[3, 0, 1], [1, 3, 0], [1, 0, 2]])) >>> y = Tensor(np.array([[0, 2, 1], [1, 2, 1], [0, 0, 1]])) >>> metric = nn.RootMeanSquareDistance(symmetric=False, distance_metric="euclidean") >>> metric.clear() >>> metric.update(x, y, 0) >>> root_mean_square_distance = metric.eval() >>> print(root_mean_square_distance) 1.0000000000000002
-
update(*inputs)[source]¶ Updates the internal evaluation result ‘y_pred’, ‘y’ and ‘label_idx’.
- Parameters
inputs –
- Input ‘y_pred’, ‘y’ and ‘label_idx’. ‘y_pred’ and ‘y’ are Tensor or numpy.ndarray. ‘y_pred’ is the
predicted binary image. ‘y’ is the actual binary image. ‘label_idx’, the data type of label_idx is int.
- Raises:
ValueError: If the number of the inputs is not 3.
-
class
tinyms.metrics.Perplexity(ignore_label=None)[source]¶ Computes perplexity. Perplexity is a measurement about how well a probability distribution or a model predicts a sample. A low perplexity indicates the model can predict the sample well. The function is shown as follows:
\[\begin{split}b^{\\big(-\\frac{1}{N} \\sum_{i=1}^N \\log_b q(x_i) \\big)} = \\exp \\big(-\\frac{1}{N} \\sum_{i=1}^N \\log q(x_i)\\big)\end{split}\]- Parameters
ignore_label (int) – Index of an invalid label to be ignored when counting. If set to None, it will include all entries. Default: -1.
Examples
>>> x = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]])) >>> y = Tensor(np.array([1, 0, 1])) >>> metric = Perplexity(ignore_label=None) >>> metric.clear() >>> metric.update(x, y) >>> perplexity = metric.eval() >>> print(perplexity) 2.231443166940565
-
eval()[source]¶ Returns the current evaluation result.
- Returns
float, the computed result.
- Raises
RuntimeError – If the sample size is 0.
-
update(*inputs)[source]¶ Updates the internal evaluation result: math:preds and :math:labels.
- Parameters
inputs – Input preds and labels. preds and labels are Tensor, list or numpy.ndarray. preds is the predicted values, labels is the label of the data. The shape of preds and labels are both \((N, C)\).
- Raises
ValueError – If the number of the inputs is not 2.
-
class
tinyms.metrics.ConfusionMatrix(num_classes, normalize='no_norm', threshold=0.5)[source]¶ Computes the confusion matrix. The performance matrix of measurement classification model is the model whose output is binary or multi class. The confusion matrix is calculated. An array of shape [BC4] is returned. The third dimension represents each channel of each sample in the input batch.Where B is the batch size and C is the number of classes to be calculated.
If you only want to find confusion matrix, use this class. If you want to find ‘PPV’, ‘TPR’, ‘TNR’, etc., use class ‘mindspore.metrics.ConfusionMatrixMetric’.
- Parameters
num_classes (int) – Number of classes in the dataset.
normalize (str) –
The parameter of calculating ConfusionMatrix supports four Normalization modes, Choose from:
’no_norm’ (None) - No normalization is used. Default: None.
’target’ (str) - Normalization based on target value.
’prediction’ (str) - Normalization based on predicted value.
’all’ (str) - Normalization over the whole matrix.
threshold (float) – A threshold, which is used to compare with the input tensor. Default: 0.5.
Examples
>>> x = Tensor(np.array([1, 0, 1, 0])) >>> y = Tensor(np.array([1, 0, 0, 1])) >>> metric = nn.ConfusionMatrix(num_classes=2, normalize="no_norm", threshold=0.5) >>> metric.clear() >>> metric.update(x, y) >>> output = metric.eval() >>> print(output) [[1. 1.] [1. 1.]]
-
update(*inputs)[source]¶ Update state with y_pred and y.
- Parameters
inputs – Input y_pred and y. y_pred and y are a Tensor, a list or an array. y_pred is the predicted value, y is the true value. The shape of y_pred is \((N, C, ...)\) or \((N, ...)\). The shape of y is \((N, ...)\).
- Raises
ValueError – If the number of the inputs is not 2.
-
class
tinyms.metrics.ConfusionMatrixMetric(skip_channel=True, metric_name='sensitivity', calculation_method=False, decrease='mean')[source]¶ The performance matrix of measurement classification model is the model whose output is binary or multi class. The correlation measure of confusion matrix was calculated from the full-scale tensor, and the average values of batch, class channel and iteration were collected. This function supports the calculation of all measures described below: the metric name in parameter metric_name.
If you want to use confusion matrix to calculate, such as ‘PPV’, ‘TPR’, ‘TNR’, use this class. If you only want to calculate confusion matrix, please use ‘mindspore.metrics.ConfusionMatrix’.
- Parameters
skip_channel (bool) – Whether to skip the measurement calculation on the first channel of the predicted output. Default: True.
metric_name (str) – The names of indicators are in the following range. Of course, you can also set the industry common aliases for these indicators. Choose from: [“sensitivity”, “specificity”, “precision”, “negative predictive value”, “miss rate”, “fall out”, “false discovery rate”, “false omission rate”, “prevalence threshold”, “threat score”, “accuracy”, “balanced accuracy”, “f1 score”, “matthews correlation coefficient”, “fowlkes mallows index”, “informedness”, “markedness”].
calculation_method (bool) – If true, the measurement for each sample is calculated first. If it is false, the confusion matrix of all samples is accumulated first. As for classification task, ‘calculation_method’ should be False. Default: False.
decrease (str) – Define the mode to reduce the calculation result of one batch of data. Decrease is used only if calculation_method is True. Default: “mean”. Choose from: [“none”, “mean”, “sum”, “mean_batch”, “sum_batch”, “mean_channel”, “sum_channel”].
Examples
>>> metric = ConfusionMatrixMetric(skip_channel=True, metric_name="tpr", ... calculation_method=False, decrease="mean") >>> metric.clear() >>> x = Tensor(np.array([[[0], [1]], [[1], [0]]])) >>> y = Tensor(np.array([[[0], [1]], [[0], [1]]])) >>> metric.update(x, y) >>> x = Tensor(np.array([[[0], [1]], [[1], [0]]])) >>> y = Tensor(np.array([[[0], [1]], [[1], [0]]])) >>> avg_output = metric.eval() >>> print(avg_output) [0.5]
-
update(*inputs)[source]¶ Update state with predictions and targets.
- inputs:
Input y_pred and y. y_pred and y are a Tensor, a list or an array.
y_pred (ndarray) - Input data to compute. It must be one-hot format and first dim is batch. The shape of y_pred is \((N, C, ...)\) or \((N, ...)\). As for classification tasks, y_pred should has the shape [BN] where N is larger than 1. As for segmentation tasks, the shape should be [BNHW] or [BNHWD].
y (ndarray) - Compute the true value of the measure. It must be one-hot format and first dim is batch. The shape of y is \((N, C, ...)\).
- Raises
ValueError – If the number of the inputs is not 2.
tinyms.hub¶
-
tinyms.hub.load(uid, pretrained=True, **kwargs)[source]¶ Load a model from remote TinyMS Hub.
- Parameters
- Returns
layers.Layer, the initialized network instance.
Examples
>>> from tinyms import hub >>> >>> hub.load('tinyms/0.2/lenet5_v1_mnist', class_num=10)
-
tinyms.hub.load_checkpoint(uid, dst, pretrained=True, **kwargs)[source]¶ Load model checkpoint file from remote TinyMS Hub.
- Parameters
uid (str) – Uid. The format should be strictly consistent with the official example: tinyms/0.2/lenet5_v1_mnist.
dst (str) – Full path of filename where the checkpoint file will be loaded, e.g. /tmp/lenet5.ckpt.
pretrained (bool) – Specified if to load pretrained weight ckpt file. Default: True.
kwargs (dict, optional) – Keyword arguments for network initialization.
Examples
>>> from tinyms import hub >>> >>> hub.load_checkpoint('tinyms/0.2/lenet5_v1_mnist', '/tmp/lenet5.ckpt', class_num=10)
-
tinyms.hub.load_weights(uid)[source]¶ Load model pretrained weights from remote TinyMS Hub.
- Parameters
uid (str) – Uid. The format should be strictly consistent with the official example: tinyms/0.2/lenet5_v1_mnist.
- Returns
Dict, the pretrained network weight dict.
Examples
>>> from tinyms import hub >>> from tinyms.model import lenet5 >>> from tinyms.utils.train import load_param_into_net >>> >>> param_dict = hub.load_weights('tinyms/0.2/lenet5_v1_mnist') >>> net = lenet5() >>> load_param_into_net(net, param_dict)
tinyms.serving¶
This module refers to the process of serving pre-trained models so that they can quickly and efficiently process data input by users and obtain results. TinyMS provides a complete set of start server (start_server), check backend (list_servables), check start status (server_started) and shut down the server (shutdown) and other functions based on Flask (https://flask.palletsprojects.com/en/1.1.x/).
Examples
>>> from tinyms.serving import Server, Client
>>>
>>> server = Server()
>>> server.start_server()
Server starts at host 127.0.0.1, port 5000
>>> client = Client()
>>> client.list_servables()
>>> client.predict('example.jpg', 'servable_name', dataset_name='mnist')
-
class
tinyms.serving.Client(host='127.0.0.1', port=5000)[source]¶ Client is the entrance to connecting to serving server, and also send all requests to server side by HTTP request.
- Parameters
Examples
>>> from tinyms.serving import Client >>> >>> client = Client()
-
list_servables()[source]¶ List the model that is currently served by the backend server.
A GET request will be sent to the server (127.0.0.1:5000) which will then be routed to 127.0.0.1:5000/servables, and the backend servable information will be returned to the client.
- Returns
res_body[‘servables’] (str) will be returned, the backend servable information. Error message will be returned and printed if requests.status_code is not ok. ‘Server not started’ will be returned if server is not started
Examples
>>> # Running the quickstart tutorial, after server started and servable json defined >>> from tinyms.serving import Client >>> >>> client = Client() >>> client.list_servables() [{ 'description': 'This servable hosts a lenet5 model predicting numbers', 'model': { 'class_num': 10, 'format': 'ckpt', 'name': 'lenet5' }, 'name': 'lenet5' }]
-
predict(img_path, servable_name, dataset_name='mnist', strategy='TOP1_CLASS')[source]¶ Send the predict request to the backend server, get the return value and do the post process
Predict the input image, and get the result. User must specify the image_path, servable_name, dataset_name and output_strategy to get the predict result.
- Parameters
img_path (str) – path to the image
servable_name (str) – the name in servable_json, now supports 6 servables: lenet5, resnet50_imagenet2012, resnet50_cifar10, mobilenetv2, ssd300 and cyclegan_cityscape.
dataset_name (str) – the name of the dataset that is used to train the model, now supports 5 datasets: mnist, imagenet2012, cifar10, voc, cityscape
strategy (str) – the output strategy, for lenet5, resnet50 and mobilenetv2, select between ‘TOP1_CLASS’ and ‘TOP5_CLASS’, for ssd300, only TOP1_CLASS, for cyclegan_cityscape, select between gray2color and color2gray
- Returns
For lenet5, resnet50, mobilenetv2, the output is a string of predict result. For ssd300, the output is a string of bounding boxes coordinates and labels, which can be further processed using ImageViewer function For cyclegan, the output is a numpy of image, which can be transformed to image using Image.fromarray
Examples
>>> # Running the quickstart tutorial, after server started and servable json defined >>> from tinyms.serving import Client >>> >>> client = Client() >>> print(client.predict('/root/7.png', 'lenet5', 'mnist', 'TOP1_CLASS')) TOP1: 7, score: 0.99943381547927856445
-
class
tinyms.serving.Server(host='127.0.0.1', port=5000, serving_path='/etc/tinyms/serving/')[source]¶ Server is the entrance to initialize and shutdown the serving server, and also accept all requests from the client side by calling Flask server.
- Parameters
Examples
>>> from tinyms.serving import Server >>> >>> server = Server()
-
shutdown()[source]¶ Shutdown the flask server.
Search fot the pid of the process running on port 5000, and kill it. This function will be automatically called when SIGINT, SIGHUP and SIGTERM signals caught.
- Returns
A string message of server shutting down or not.
Examples
>>> from tinyms.serving import Server >>> >>> server = Server() >>> server.shutdown() Server shutting down...
-
start_server()[source]¶ Start the flask server in a subprocess.
Catch the signal of CTRL + D to shutdown, otherwise call shutdown() function to shutdown the server, if the ip and port already in use, server won’t start for a second time.
- Returns
Start the server in a sub process.
Examples
>>> from tinyms.serving import Server >>> >>> server = Server() >>> server.start_server() Server starts at host 127.0.0.1, port 5000
-
tinyms.serving.run_flask(host='127.0.0.1', port=5000)[source]¶ Start the flask server, only be used to trigger starting the flask server in subprocess.
Directly calling this function is not recommended, please use start_server(). Only Error message will be displayed.
- Parameters
- Returns
Server Started
Examples
>>> # In the start_server function >>> cmd = ['python -c "from tinyms.serving import run_flask; run_flask()"'] >>> server_process = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
Release Roadmap¶
v0.2.0¶
Major Features Planning¶
RNN support with initial focus on LSTM and SentimentNet
ModelPark additions: Alex,Densenet100,Bert.
TinyMS Hub, for one click pretrained model depolyment and inference.
Adding TinyMS into MindSpore CI pipeline to ensure no breaking change to affect high level API experience.
Plan will be finalized at the upcoming community design meeting.
Contributing Guidelines¶
Firstly a great welcome for anyone who wants to participate in TinyMS community👏. For those who are not familiar with how this community works, these are some guidelines to get you quickly getting started.
Contributor License Agreement¶
It’s required to sign CLA before your first code submission to TinyMS community.
For individual contributor, please refer to ICLA sign page for the detailed information.
Getting Started¶
Fork the repository on GitHub.
Read the README.md and install page for project information and build instructions.
Try our first quick start tutorial in one minutes!😍
Contribution Workflow¶
Code style¶
Please follow this style to make TinyMS easy to review, maintain and develop.
Coding guidelines
The Python coding style suggested by Python PEP 8 Coding Style is adopted in TinyMS community.
Unittest guidelines
The Python unittest style suggested by pytest is adopted in TinyMS community.
Autodoc guidelines
The Autodoc generated style suggested by Sphinx is adopted in TinyMS community.
Fork-Pull development model¶
Fork TinyMS repository
Before submitting code to TinyMS project, please make sure that this project have been forked to your own repository. It means that there will be parallel development between TinyMS repository and your own repository, so be careful to avoid the inconsistency between them.
NOTICE: The default branch name of TinyMS project is
maininstead ofmaster.Clone the remote repository
If you want to download the code to the local machine,
gitis the best choice:git clone https://github.com/{insert_your_forked_repo}/tinyms.git git remote add upstream https://github.com/tinyms-ai/tinyms.git
Develop code locally
To avoid inconsistency between multiple branches, checking out to a new branch for every pull request is
SUGGESTED:git checkout -b {new_branch_name}
NOTICE: Please try to pull the latest code from upstream repository (
git pull upstream main) every time before checking out a new branch.Then you can change the code arbitrarily.
Push the code to the remote repository
After updating the code, you should push the update in the formal way:
git add . git status # Check the update status git commit -m "Your commit title" git commit -s --amend # Add the concrete description of your commit git push origin {new_branch_name}
Pull a request to TinyMS repository
In the last step, your need to pull a compare request between your new branch and TinyMS
mainbranch. After finishing the pull request, the Travis CI will be automatically set up for building test.
Report issues¶
A great way to contribute to the project is to send a detailed report when you encounter an issue. We always appreciate a well-written, thorough bug report, and will thank you for it!🤝
When reporting issues, refer to this format:
What version of env (tinyms, mindspore, os, python etc) are you using?
Is this a BUG REPORT or FEATURE REQUEST?
What happened?
What you expected to happen?
How to reproduce it? (as minimally and precisely as possible)
Special notes for your reviewers?
Issues advisory:
If you find an unclosed issue, which is exactly what you are going to solve, please put some comments on that issue to tell others you would be in charge of it.
If an issue is opened for a while, it’s recommended for contributors to precheck before working on solving that issue.
If you resolve an issue which is reported by yourself, it’s also required to let others know before closing that issue.
Propose PRs¶
Working on your first Pull Request? 📚You can learn how from this free series How to Contribute to an Open Source Project on GitHub📚
When proposing pull requests, please adhere to these rules:
Raise your idea as an issue on GitHub.
If it is a new feature that needs lots of design details, a design proposal should also be submitted.
After reaching consensus in the issue discussions and design proposal reviews, complete the development on the forked repo and submit a PR.
None of PRs is not permitted until it receives 2+ LGTM from approvers. Please NOTICE that approver is NOT allowed to add LGTM on his own PR.
After PR is sufficiently discussed, it will get merged, abandoned or rejected depending on the outcome of the discussion.
PRs advisory:
Any irrelevant changes should be avoided.
Make sure your commit history being ordered.
Always keep your branch up with the master branch.
For bug-fix PRs, make sure all related issues being linked.
Community Support¶
Whenever you feel confused in this community, please be free to reach out the help from TinyMS community with these approaches:
Wechat communication. Add
mindspore0328Wechat ID to ask for help.QQ group. TBD.
Slack channel. Join
tinymschannel in MindSpore Slack to communicate with each other.
Communication¶
Technical Discussion¶
ISSUEs and PRs are always the preferred way to have technical discussions. Please refer to the guidelines
Community Support¶
Whenever you feel confused in this community, please be free to reach out the help from TinyMS community with these approaches:
Wechat communication. Add
mindspore0328Wechat ID to ask for help.QQ group. TBD.
Slack channel. Join
tinymschannel in MindSpore Slack to communicate with each other.
FAQ¶
TODO