tinyms.losses¶

Losses module. Loss function in machine learning is the target of the model. It shows how well the model works on a dataset and the optimization target which the optimizer is searching.

tinyms.losses.net_with_loss(net)[source]¶

This function is provided for AI beginners who are not familiar with which loss should be chosen for the network to be trained. Instead of choosing different loss function, users could directly get the best suitable loss function by specifying network.

Parameters: net (layers.Layer) – The instance of network to be trained.
Raises: TypeError – When network type is not supported.

Note

Currently this function only supports few networks, if the network type is not supported, the system would raise TypeError exception.

Examples

>>> from tinyms.model import ssd300
>>> from tinyms.losses import net_with_loss
>>>
>>> net = ssd300()
>>> net_loss = net_with_loss(net)

class tinyms.losses.SSD300WithLoss(network)[source]¶

Provide SSD300 training loss through network.

Parameters: network (layers.Layer) – The training network.
Returns: Tensor, the loss of the network.

Examples

>>> from tinyms.model import ssd300
>>> from tinyms.losses import SSD300WithLoss
>>>
>>> net = SSD300WithLoss(ssd300())

class tinyms.losses.CrossEntropyWithLabelSmooth(smooth_factor=0.0, num_classes=1000)[source]¶

CrossEntropyWith LabelSmooth.

Parameters

smooth_factor (float) – Smooth factor. Default is 0.
num_classes (int) – Number of classes. Default is 1000.

Returns

None.

Examples

>>> CrossEntropyWithLabelSmooth(smooth_factor=0., num_classes=1000)

class tinyms.losses.CycleGANGeneratorLoss(generator, D_A, D_B)[source]¶

Cycle GAN generator loss.

Parameters

generator (layers.Layer) – Generator of CycleGAN.
D_A (layers.Layer) – The discriminator network of domain A to domain B.
D_B (layers.Layer) – The discriminator network of domain B to domain A.

Outputs:: Tuple Tensor, the losses of generator.

construct(img_A, img_B)[source]¶: If use_identity, identity loss will be used.

class tinyms.losses.CycleGANDiscriminatorLoss(D_A, D_B, reduction='none')[source]¶

Cycle GAN discriminator loss.

Parameters

D_A (layers.Layer) – The discriminator network of domain A to domain B.
D_B (layers.Layer) – The discriminator network of domain B to domain A.
reduction (str) – The discriminator network of reduction. Default: none.

Outputs:: the loss of discriminator.

class tinyms.losses.L1Loss(reduction='mean')[source]¶

L1Loss creates a criterion to measure the mean absolute error (MAE) between \(x\) and \(y\) element-wise, where \(x\) is the input Tensor and \(y\) is the target Tensor.

For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:

\[L(x, y) = \{l_1,\dots,l_N\}, \quad \text{with } l_n = \left| x_n - y_n \right|\]

When argument reduction is ‘mean’, the mean value of \(L(x, y)\) will be returned. When argument reduction is ‘sum’, the sum of \(L(x, y)\) will be returned. \(N\) is the batch size.

Parameters: reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.

Inputs:

input_data (Tensor) - Tensor of shape \((x_1, x_2, ..., x_R)\). The data type must be float16 or float32.
target_data (Tensor) - Tensor of shape \((y_1, y_2, ..., y_S)\). The data type must be float16 or float32.

Outputs:

Tensor, loss float tensor.

Supported Platforms:

Ascend GPU

Examples

>>> loss = nn.L1Loss()
>>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32)
>>> output = loss(input_data, target_data)
>>> print(output)
0.33333334

class tinyms.losses.MSELoss(reduction='mean')[source]¶

MSELoss creates a criterion to measure the mean squared error (squared L2-norm) between \(x\) and \(y\) element-wise, where \(x\) is the input and \(y\) is the target.

For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:

\[L(x, y) = \{l_1,\dots,l_N\}, \quad \text{with} \quad l_n = (x_n - y_n)^2.\]

When argument reduction is ‘mean’, the mean value of \(L(x, y)\) will be returned. When argument reduction is ‘sum’, the sum of \(L(x, y)\) will be returned. \(N\) is the batch size.

Parameters: reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.

Inputs:

input_data (Tensor) - Tensor of shape \((x_1, x_2, ..., x_R)\).
target_data (Tensor) - Tensor of shape \((y_1, y_2, ..., y_S)\).

Outputs:

Tensor, weighted loss float tensor.

Supported Platforms:

Ascend GPU

Examples

>>> loss = nn.MSELoss()
>>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32)
>>> output = loss(input_data, target_data)
>>> print(output)
0.33333334

class tinyms.losses.SmoothL1Loss(beta=1.0)[source]¶

A loss class for learning region proposals.

SmoothL1Loss can be regarded as modified version of L1Loss or a combination of L1Loss and L2Loss. L1Loss computes the element-wise absolute difference between two input Tensor while L2Loss computes the squared difference between two input Tensor. L2Loss often leads to faster convergence but it is less robust to outliers.

Given two input \(x,\ y\) of length \(N\), the unreduced SmoothL1Loss can be described as follows:

\[\begin{split}L_{i} = \begin{cases} \frac{0.5 (x_i - y_i)^{2}}{\text{beta}}, & \text{if } |x_i - y_i| < \text{beta} \\ |x_i - y_i| - 0.5 \text{beta}, & \text{otherwise. } \end{cases}\end{split}\]

Here \(\text{beta}\) controls the point where the loss function changes from quadratic to linear. Its default value is 1.0. \(N\) is the batch size. This function returns an unreduced loss Tensor.

Parameters: beta (float) – A parameter used to control the point where the function will change from quadratic to linear. Default: 1.0.

Inputs:

input_data (Tensor) - Tensor of shape \((x_1, x_2, ..., x_R)\).
target_data (Tensor) - Tensor of shape \((y_1, y_2, ..., y_S)\).

Outputs:

Tensor, loss float tensor.

Supported Platforms:

Ascend GPU CPU

Examples

>>> loss = nn.SmoothL1Loss()
>>> input_data = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> target_data = Tensor(np.array([1, 2, 2]), mindspore.float32)
>>> output = loss(input_data, target_data)
>>> print(output)
[0.  0.  0.5]

class tinyms.losses.SoftmaxCrossEntropyWithLogits(sparse=False, reduction='none')[source]¶

Computes softmax cross entropy between logits and labels.

Measures the distribution error between the probabilities of the input (computed with softmax function) and the target where the classes are mutually exclusive (only one class is positive) using cross entropy loss.

Typical input into this function is unnormalized scores and target of each class. Scores Tensor \(x\) is of shape \((N, C)\) and target Tensor \(t\) is a Tensor of shape \((N, C)\) which contains one-hot labels of length \(C\).

For each instance \(N_i\), the loss is given as:

\[\ell(x_i, t_i) = - \log\left(\frac{\exp(x_{t_i})}{\sum_j \exp(x_j)}\right) = -x_{t_i} + \log\left(\sum_j \exp(x_j)\right)\]

where \(x_i\) is a 1D score Tensor, \(t_i\) is a scalar.

Note

While the target classes are mutually exclusive, i.e., only one class is positive in the target, the predicted probabilities need not to be exclusive. It is only required that the predicted probability distribution of entry is a valid one.

Parameters

sparse (bool) – Specifies whether labels use sparse format or not. Default: False.
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. If “none”, do not perform reduction. Default: “none”.

Inputs:

logits (Tensor) - Tensor of shape (N, C).
labels (Tensor) - Tensor of shape (N, ). If sparse is True, The type of labels is mindspore.int32. If sparse is False, the type of labels is the same as the type of logits.

Outputs:

Tensor, a tensor of the same shape as logits with the component-wise logistic losses.

Supported Platforms:

Ascend GPU CPU

Examples

>>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True)
>>> np.random.seed(0)
>>> logits = Tensor(np.random.randint(0, 9, [1, 10]), mindspore.float32)
>>> labels_np = np.ones([1,]).astype(np.int32)
>>> labels = Tensor(labels_np)
>>> output = loss(logits, labels)
>>> print(output)
[7.868383]

class tinyms.losses.BCELoss(weight=None, reduction='none')[source]¶

BCELoss creates a criterion to measure the binary cross entropy between the true labels and predicted labels.

Set the predicted labels as \(x\), true labels as \(y\), the output loss as \(\ell(x, y)\). Let,

\[L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log x_n + (1 - y_n) \cdot \log (1 - x_n) \right]\]

Then,

\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{`none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{`mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{`sum'.} \end{cases}\end{split}\]

Note

Note that the predicted labels should always be the output of sigmoid and the true labels should be numbers between 0 and 1.

Parameters

weight (Tensor, optional) – A rescaling weight applied to the loss of each batch element. And it must have same shape and data type as inputs. Default: None
reduction (str) – Specifies the reduction to be applied to the output. Its value must be one of ‘none’, ‘mean’, ‘sum’. Default: ‘none’.

Inputs:

inputs (Tensor) - The input Tensor. The data type must be float16 or float32.
labels (Tensor) - The label Tensor which has same shape and data type as inputs.

Outputs:

Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as inputs. Otherwise, the output is a scalar.

Supported Platforms:

Ascend GPU

Examples

>>> weight = Tensor(np.array([[1.0, 2.0, 3.0], [4.0, 3.3, 2.2]]), mindspore.float32)
>>> loss = nn.BCELoss(weight=weight, reduction='mean')
>>> inputs = Tensor(np.array([[0.1, 0.2, 0.3], [0.5, 0.7, 0.9]]), mindspore.float32)
>>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]), mindspore.float32)
>>> output = loss(inputs, labels)
>>> print(output)
1.8952923

class tinyms.losses.CosineEmbeddingLoss(margin=0.0, reduction='mean')[source]¶

Computes the similarity between two tensors using cosine distance.

Given two tensors x1, x2, and a Tensor label y with values 1 or -1:

\[\begin{split}loss(x_1, x_2, y) = \begin{cases} 1-cos(x_1, x_2), & \text{if } y = 1\\ max(0, cos(x_1, x_2)-margin), & \text{if } y = -1\\ \end{cases}\end{split}\]

Parameters

margin (float) – Should be in [-1.0, 1.0]. Default 0.0.
reduction (str) – Specifies which reduction to be applied to the output. It must be one of “none”, “mean”, and “sum”, meaning no reduction, reduce mean and sum on output, respectively. Default “mean”.

Inputs:

input_x1 (Tensor) - Input tensor.
input_x2 (Tensor) - Its shape and data type must be the same as input_x1’s shape and data type.
y (Tensor) - Contains value 1 or -1. Suppose the shape of input_x1 is \((x_1, x_2, x_3,..., x_R)\), then the shape of target must be \((x_1, x_3, x_4, ..., x_R)\).

Outputs:

loss (Tensor) - If reduction is “none”, its shape is the same as y’s shape, otherwise a scalar value will be returned.

Supported Platforms:

Ascend GPU

Examples

>>> x1 = Tensor(np.array([[0.3, 0.8], [0.4, 0.3]]), mindspore.float32)
>>> x2 = Tensor(np.array([[0.4, 1.2], [-0.4, -0.9]]), mindspore.float32)
>>> y = Tensor(np.array([1,-1]), mindspore.int32)
>>> cosine_embedding_loss = nn.CosineEmbeddingLoss()
>>> output = cosine_embedding_loss(x1, x2, y)
>>> print(output)
0.0003426075

class tinyms.losses.SampledSoftmaxLoss(num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=True, seed=0, reduction='none')[source]¶

Computes the sampled softmax training loss.

Parameters

num_sampled (int) – The number of classes to randomly sample per batch.
num_classes (int) – The number of possible classes.
num_true (int) – The number of target classes per training example.
sampled_values (Tuple) – Tuple of (sampled_candidates, true_expected_count, sampled_expected_count) returned by a *CandidateSampler function. Default to None, UniformCandidateSampler is applied.
remove_accidental_hits (bool) – Whether to remove “accidental hits” where a sampled class equals one of the target classes. Default is True.
seed (int) – Random seed for candidate sampling. Default: 0
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. If “none”, do not perform reduction. Default: “none”.

Inputs:

weights (Tensor) - Tensor of shape (C, dim).
bias (Tensor) - Tensor of shape (C). The class biases.
labels (Tensor) - Tensor of shape (N, num_true), type int64, int32. The target classes.
inputs (Tensor) - Tensor of shape (N, dim). The forward activations of the input network.

Outputs:

Tensor, a tensor of shape (N) with the per-example sampled softmax losses.

Supported Platforms:

GPU

Examples

>>> mindspore.set_seed(1)
>>> loss = nn.SampledSoftmaxLoss(num_sampled=4, num_classes=7, num_true=1)
>>> weights = Tensor(np.random.randint(0, 9, [7, 10]), mindspore.float32)
>>> biases = Tensor(np.random.randint(0, 9, [7]), mindspore.float32)
>>> labels = Tensor([0, 1, 2])
>>> inputs = Tensor(np.random.randint(0, 9, [3, 10]), mindspore.float32)
>>> output = loss(weights, biases, labels, inputs)
>>> print(output)
[4.6051701e+01 1.4000047e+01 6.1989022e-06]