tinyms.losses¶

Losses module. Loss function in machine learning is the target of the model. It shows how well the model works on a dataset and the optimization target which the optimizer is searching.

tinyms.losses.net_with_loss(net)[source]¶

This function is provided for AI beginners who are not familiar with which loss should be chosen for the network to be trained. Instead of choosing different loss function, users could directly get the best suitable loss function by specifying network.

Parameters: net (layers.Layer) – The instance of network to be trained.
Raises: TypeError – When network type is not supported.

Note

Currently this function only supports few networks, if the network type is not supported, the system would raise TypeError exception.

Examples

>>> from tinyms.model import ssd300
>>> from tinyms.losses import net_with_loss
>>>
>>> net = ssd300()
>>> net_loss = net_with_loss(net)

class tinyms.losses.SSD300WithLoss(network)[source]¶

Provide SSD300 training loss through network.

Parameters: network (layers.Layer) – The training network.
Returns: Tensor, the loss of the network.

Examples

>>> from tinyms.model import ssd300
>>> from tinyms.losses import SSD300WithLoss
>>>
>>> net = SSD300WithLoss(ssd300())

class tinyms.losses.CrossEntropyWithLabelSmooth(smooth_factor=0.0, num_classes=1000)[source]¶

CrossEntropyWith LabelSmooth.

Parameters

smooth_factor (float) – Smooth factor. Default is 0.
num_classes (int) – Number of classes. Default is 1000.

Returns

None.

Examples

>>> CrossEntropyWithLabelSmooth(smooth_factor=0., num_classes=1000)

class tinyms.losses.CycleGANGeneratorLoss(generator, D_A, D_B)[source]¶

Cycle GAN generator loss.

Parameters

generator (layers.Layer) – Generator of CycleGAN.
D_A (layers.Layer) – The discriminator network of domain A to domain B.
D_B (layers.Layer) – The discriminator network of domain B to domain A.

Outputs:: Tuple Tensor, the losses of generator.

construct(img_A, img_B)[source]¶: If use_identity, identity loss will be used.

class tinyms.losses.CycleGANDiscriminatorLoss(D_A, D_B, reduction='none')[source]¶

Cycle GAN discriminator loss.

Parameters

D_A (layers.Layer) – The discriminator network of domain A to domain B.
D_B (layers.Layer) – The discriminator network of domain B to domain A.
reduction (str) – The discriminator network of reduction. Default: none.

Outputs:: the loss of discriminator.

class tinyms.losses.LossBase(reduction='mean')[source]¶

Base class for other losses.

Other losses derived from this should implement their own construct and use method self.get_loss to apply reduction to loss values.

Parameters: reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.
Raises: ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

Supported Platforms:: Ascend GPU CPU

get_axis(x)[source]¶

Get a range of axis for input.

Parameters: x (Tensor) – Tensor of any shape.

get_loss(x, weights=1.0)[source]¶

Computes the weighted loss.

Parameters

x (Tensor) – Tensor of shape \((N, *)\) where \(*\) means, any number of additional dimensions.
weights (Union[float, Tensor]) – Optional Tensor whose rank is either 0, or the same rank as inputs, and must be broadcastable to inputs (i.e., all dimensions must be either 1, or the same as the corresponding inputs dimension).

class tinyms.losses.L1Loss(reduction='mean')[source]¶

L1Loss creates a criterion to measure the mean absolute error (MAE) between \(x\) and \(y\) element-wise, where \(x\) is the input Tensor and \(y\) is the labels Tensor.

For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:

\[\ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad \text{with } l_n = \left| x_n - y_n \right|,\]

where \(N\) is the batch size. If reduction is not ‘none’, then:

\[\begin{split}\ell(x, y) = \begin{cases} \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]

Parameters: reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.

Inputs:

logits (Tensor) - Tensor of shape \((N, *)\) where \(*\) means, any number of additional dimensions.
labels (Tensor) - Tensor of shape \((N, *)\), same shape as the logits in common cases. However, it supports the shape of logits is different from the shape of labels and they should be broadcasted to each other.

Outputs:

Tensor, loss float tensor, the shape is zero if reduction is ‘mean’ or ‘sum’, while the shape of output is the broadcasted shape if reduction is ‘none’.

Raises: ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

Supported Platforms:: Ascend GPU CPU

Examples

>>> # Case 1: logits.shape = labels.shape = (3,)
>>> loss = nn.L1Loss()
>>> logits = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> labels = Tensor(np.array([1, 2, 2]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
0.33333334
>>> # Case 2: logits.shape = (3,), labels.shape = (2, 3)
>>> loss = nn.L1Loss(reduction='none')
>>> logits = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> labels = Tensor(np.array([[1, 1, 1], [1, 2, 2]]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
[[0. 1. 2.]
 [0. 0. 1.]]

class tinyms.losses.MSELoss(reduction='mean')[source]¶

MSELoss creates a criterion to measure the mean squared error (squared L2-norm) between \(x\) and \(y\) element-wise, where \(x\) is the input and \(y\) is the labels.

For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:

\[\ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad \text{with} \quad l_n = (x_n - y_n)^2.\]

where \(N\) is the batch size. If reduction is not ‘none’, then:

\[\begin{split}\ell(x, y) = \begin{cases} \operatorname{mean}(L), & \text{if reduction} = \text{`mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{`sum'.} \end{cases}\end{split}\]

Parameters: reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.

Inputs:

logits (Tensor) - Tensor of shape \((N, *)\) where \(*\) means, any number of additional dimensions.
labels (Tensor) - Tensor of shape \((N, *)\), same shape as the logits in common cases. However, it supports the shape of logits is different from the shape of labels and they should be broadcasted to each other.

Outputs:

Tensor, loss float tensor, the shape is zero if reduction is ‘mean’ or ‘sum’, while the shape of output is the broadcasted shape if reduction is ‘none’.

Raises: ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

Supported Platforms:: Ascend GPU CPU

Examples

>>> # Case 1: logits.shape = labels.shape = (3,)
>>> loss = nn.MSELoss()
>>> logits = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> labels = Tensor(np.array([1, 1, 1]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
1.6666667
>>> # Case 2: logits.shape = (3,), labels.shape = (2, 3)
>>> loss = nn.MSELoss(reduction='none')
>>> logits = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> labels = Tensor(np.array([[1, 1, 1], [1, 2, 2]]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
[[0. 1. 4.]
 [0. 0. 1.]]

class tinyms.losses.SmoothL1Loss(beta=1.0)[source]¶

A loss class for learning region proposals.

SmoothL1Loss can be regarded as modified version of L1Loss or a combination of L1Loss and L2Loss. L1Loss computes the element-wise absolute difference between two input tensors while L2Loss computes the squared difference between two input tensors. L2Loss often leads to faster convergence but it is less robust to outliers.

Given two input \(x,\ y\) of length \(N\), the unreduced SmoothL1Loss can be described as follows:

\[\begin{split}L_{i} = \begin{cases} \frac{0.5 (x_i - y_i)^{2}}{\text{beta}}, & \text{if } |x_i - y_i| < \text{beta} \\ |x_i - y_i| - 0.5 \text{beta}, & \text{otherwise. } \end{cases}\end{split}\]

Here \(\text{beta}\) controls the point where the loss function changes from quadratic to linear. Its default value is 1.0. \(N\) is the batch size. This function returns an unreduced loss tensor.

Parameters: beta (float) – A parameter used to control the point where the function will change from quadratic to linear. Default: 1.0.

Inputs:

logits (Tensor) - Tensor of shape \((N, *)\) where \(*\) means, any number of additional dimensions. Data type must be float16 or float32.
labels (Tensor) - Ground truth data, tensor of shape \((N, *)\), same shape and dtype as the logits.

Outputs:

Tensor, loss float tensor, same shape and dtype as the logits.

Raises

TypeError – If beta is not a float.
TypeError – If dtype of logits or labels is neither float16 not float32.
ValueError – If beta is less than or equal to 0.
ValueError – If shape of logits is not the same as labels.

Supported Platforms:: Ascend GPU CPU

Examples

>>> loss = nn.SmoothL1Loss()
>>> logits = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> labels = Tensor(np.array([1, 2, 2]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
[0.  0.  0.5]

class tinyms.losses.SoftMarginLoss(reduction='mean')[source]¶

A loss class for two-class classification problems.

SoftMarginLoss creates a criterion that optimizes a two-class classification logistic loss between input tensor \(x\) and labels tensor \(y\) (containing 1 or -1).

\[\text{loss}(x, y) = \sum_i \frac{\log(1 + \exp(-y[i]*x[i]))}{\text{x.nelement}()}\]

Parameters: reduction (str) – Apply specific reduction method to the output: ‘none’, ‘mean’, ‘sum’. Default: “mean”.

Inputs:

logits (Tensor) - Predict data. Data type must be float16 or float32.
labels (Tensor) - Ground truth data, with the same type and shape as logits.

Outputs:

Tensor or Scalar, if reduction is “none”, its shape is the same as logits. Otherwise, a scalar value will be returned.

Raises

TypeError – If logits or labels is not a Tensor.
TypeError – If dtype of logits or labels is neither float16 nor float32.
ValueError – If shape of logits is not the same as labels.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

Supported Platforms:: Ascend

Examples

>>> loss = ops.SoftMarginLoss()
>>> logits = Tensor(np.array([[0.3, 0.7], [0.5, 0.5]]), mindspore.float32)
>>> labels = Tensor(np.array([[-1, 1], [1, -1]]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
0.6764238

class tinyms.losses.FocalLoss(weight=None, gamma=2.0, reduction='mean')[source]¶

The loss function proposed by Kaiming team in their paper Focal Loss for Dense Object Detection improves the effect of image object detection. It is a loss function to solve the imbalance of categories and the difference of classification difficulty. If you want to learn more, please refer to the paper. Focal Loss for Dense Object Detection. The function is shown as follows:

\[FL(p_t) = -(1-p_t)^\gamma log(p_t)\]

Parameters

gamma (float) – Gamma is used to adjust the steepness of weight curve in focal loss. Default: 2.0.
weight (Union[Tensor, None]) – A rescaling weight applied to the loss of each batch element. The dimension of weight should be 1. If None, no weight is applied. Default: None.
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. If “none”, do not perform reduction. Default: “mean”.

Inputs:

logits (Tensor) - Tensor of shape should be \((B, C)\) or \((B, C, H)\) or \((B, C, H, W)\). Where \(C\) is the number of classes. Its value is greater than 1. If the shape is \((B, C, H, W)\) or \((B, C, H)\), the \(H\) or product of \(H\) and \(W\) should be the same as labels.
labels (Tensor) - Tensor of shape should be \((B, C)\) or \((B, C, H)\) or \((B, C, H, W)\). The value of \(C\) is 1 or it needs to be the same as predict’s \(C\). If \(C\) is not 1, the shape of target should be the same as that of predict, where \(C\) is the number of classes. If the shape is \((B, C, H, W)\) or \((B, C, H)\), the \(H\) or product of \(H\) and \(W\) should be the same as logits.

Outputs:

Tensor or Scalar, if reduction is “none”, its shape is the same as logits. Otherwise, a scalar value will be returned.

Raises

TypeError – If the data type of gamma is not a float.
TypeError – If weight is not a Tensor.
ValueError – If labels dim is different from logits.
ValueError – If labels channel is not 1 and labels shape is different from logits.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

Supported Platforms:: Ascend

Example

>>> logits = Tensor([[0.8, 1.4], [0.5, 0.9], [1.2, 0.9]], mstype.float32)
>>> labels = Tensor([[1], [1], [0]], mstype.int32)
>>> focalloss = nn.FocalLoss(weight=Tensor([1, 2]), gamma=2.0, reduction='mean')
>>> output = focalloss(logits, labels)
>>> print(output)
0.12516622

class tinyms.losses.SoftmaxCrossEntropyWithLogits(sparse=False, reduction='none')[source]¶

Computes softmax cross entropy between logits and labels.

Measures the distribution error between the probabilities of the input (computed with softmax function) and the labels where the classes are mutually exclusive (only one class is positive) using cross entropy loss.

Typical input into this function is unnormalized scores denoted as x whose shape is (N, C), and the corresponding targets.

For each instance \(x_i\), i ranges from 0 to N-1, the loss is given as:

\[\ell(x_i, c) = - \log\left(\frac{\exp(x_i[c])}{\sum_j \exp(x_i[j])}\right) = -x_i[c] + \log\left(\sum_j \exp(x_i[j])\right)\]

where \(x_i\) is a 1D score Tensor, \(c\) is the index of 1 in one-hot.

Note

While the labels classes are mutually exclusive, i.e., only one class is positive in the labels, the predicted probabilities does not need to be exclusive. It is only required that the predicted probability distribution of entry is a valid one.

Parameters

sparse (bool) – Specifies whether labels use sparse format or not. Default: False.
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. If “none”, do not perform reduction. Default: “none”.

Inputs:

logits (Tensor) - Tensor of shape (N, C). Data type must be float16 or float32.
labels (Tensor) - Tensor of shape (N, ). If sparse is True, The type of labels is int32 or int64. Otherwise, the type of labels is the same as the type of logits.

Outputs:

Tensor, a tensor of the same shape and type as logits with the component-wise logistic losses.

Raises

TypeError – If sparse is not a bool.
TypeError – If sparse is True and dtype of labels is neither int32 not int64.
TypeError – If sparse is False and dtype of labels is neither float16 not float32.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

Supported Platforms:: Ascend GPU CPU

Examples

>>> # case 1: sparse=True
>>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True)
>>> logits = Tensor(np.array([[3, 5, 6, 9, 12, 33, 42, 12, 32, 72]]), mindspore.float32)
>>> labels_np = np.array([1]).astype(np.int32)
>>> labels = Tensor(labels_np)
>>> output = loss(logits, labels)
>>> print(output)
[67.]
>>> # case 2: sparse=False
>>> loss = nn.SoftmaxCrossEntropyWithLogits(sparse=False)
>>> logits = Tensor(np.array([[3, 5, 6, 9, 12, 33, 42, 12, 32, 72]]), mindspore.float32)
>>> labels_np = np.array([[0, 0, 0, 0, 0, 0, 1, 0, 0, 0]]).astype(np.float32)
>>> labels = Tensor(labels_np)
>>> output = loss(logits, labels)
>>> print(output)
[30.]

class tinyms.losses.BCELoss(weight=None, reduction='none')[source]¶

BCELoss creates a criterion to measure the binary cross entropy between the true labels and predicted labels.

Set the predicted labels as \(x\), true labels as \(y\), the output loss as \(\ell(x, y)\). Let,

\[L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log x_n + (1 - y_n) \cdot \log (1 - x_n) \right]\]

Then,

\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]

Note

Note that the predicted labels should always be the output of sigmoid and the true labels should be numbers between 0 and 1.

Parameters

weight (Tensor, optional) – A rescaling weight applied to the loss of each batch element. And it must have the same shape and data type as inputs. Default: None
reduction (str) – Specifies the reduction to be applied to the output. Its value must be one of ‘none’, ‘mean’, ‘sum’. Default: ‘none’.

Inputs:

logits (Tensor) - The input tensor with shape \((N, *)\) where \(*\) means, any number of additional dimensions. The data type must be float16 or float32.
labels (Tensor) - The label tensor with shape \((N, *)\), the same shape and data type as logits.

Outputs:

Tensor or Scalar, if reduction is ‘none’, then output is a tensor and has the same shape as logits. Otherwise, the output is a scalar.

Raises

TypeError – If dtype of logits, labels or weight (if given) is neither float16 not float32.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
ValueError – If shape of logits is not the same as labels or weight (if given).

Supported Platforms:: Ascend GPU CPU

Examples

>>> weight = Tensor(np.array([[1.0, 2.0, 3.0], [4.0, 3.3, 2.2]]), mindspore.float32)
>>> loss = nn.BCELoss(weight=weight, reduction='mean')
>>> logits = Tensor(np.array([[0.1, 0.2, 0.3], [0.5, 0.7, 0.9]]), mindspore.float32)
>>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
1.8952923

class tinyms.losses.BCEWithLogitsLoss(reduction='mean', weight=None, pos_weight=None)[source]¶

Adds sigmoid activation function to input logits, and uses the given logits to compute binary cross entropy between the logits and the labels.

Sets input logits as \(X\), input labels as \(Y\), output as \(L\). Then,

\[p_{ij} = sigmoid(X_{ij}) = \frac{1}{1 + e^{-X_{ij}}}\]

\[L_{ij} = -[Y_{ij} \cdot log(p_{ij}) + (1 - Y_{ij}) \cdot log(1 - p_{ij})]\]

Then,

\[\begin{split}\ell(x, y) = \begin{cases} L, & \text{if reduction} = \text{'none';}\\ \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]

Parameters

reduction (str) – Type of reduction to be applied to loss. The optional values are ‘mean’, ‘sum’, and ‘none’. If ‘none’, do not perform reduction. Default:’mean’.
weight (Tensor, optional) – A rescaling weight applied to the loss of each batch element. If not None, it can be broadcast to a tensor with shape of logits, data type must be float16 or float32. Default: None.
pos_weight (Tensor, optional) – A weight of positive examples. Must be a vector with length equal to the number of classes. If not None, it must can be broadcast to a tensor with shape of logits, data type must be float16 or float32. Default: None.

Inputs:

logits (Tensor) - Input logits with shape \((N, *)\) where \(*\) means, any number of additional dimensions. The data type must be float16 or float32.
labels (Tensor) - Ground truth label with shape \((N, *)\), same shape and dtype as logits.

Outputs:

Tensor or Scalar, if reduction is “none”, its shape is the same as logits. Otherwise, a scalar value will be returned.

Raises

TypeError – If data type of logits or labels is neither float16 nor float32.
TypeError – If weight or pos_weight is a parameter.
TypeError – If data type of weight or pos_weight is neither float16 nor float32.
ValueError – If weight or pos_weight can not be broadcast to a tensor with shape of logits.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

Supported Platforms:: Ascend GPU

Examples

>>> logits = Tensor(np.array([[-0.8, 1.2, 0.7], [-0.1, -0.4, 0.7]]).astype(np.float32))
>>> labels = Tensor(np.array([[0.3, 0.8, 1.2], [-0.6, 0.1, 2.2]]).astype(np.float32))
>>> loss = nn.BCEWithLogitsLoss()
>>> output = loss(logits, labels)
>>> print(output)
0.3463612

class tinyms.losses.CosineEmbeddingLoss(margin=0.0, reduction='mean')[source]¶

CosineEmbeddingLoss creates a criterion to measure the similarity between two tensors using cosine distance.

Given two tensors \(x1\), \(x2\), and a Tensor label \(y\) with values 1 or -1:

\[\begin{split}loss(x_1, x_2, y) = \begin{cases} 1-cos(x_1, x_2), & \text{if } y = 1\\ max(0, cos(x_1, x_2)-margin), & \text{if } y = -1\\ \end{cases}\end{split}\]

Parameters

margin (float) – Should be in [-1.0, 1.0]. Default 0.0.
reduction (str) – Specifies which reduction to be applied to the output. It must be one of “none”, “mean”, and “sum”, meaning no reduction, reduce mean and sum on output, respectively. Default “mean”.

Inputs:

logits_x1 (Tensor) - Tensor of shape \((N, *)\) where \(*\) means, any number of additional dimensions.
logits_x2 (Tensor) - Tensor of shape \((N, *)\), same shape and dtype as logits_x1.
labels (Tensor) - Contains value 1 or -1. Suppose the shape of logits_x1 is \((x_1, x_2, x_3, ..., x_R)\), then the shape of labels must be \((x_1, x_3, x_4, ..., x_R)\).

Outputs:

Tensor or Scalar, if reduction is “none”, its shape is the same as labels. Otherwise, a scalar value will be returned.

Raises

TypeError – If margin is not a float.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
ValueError – If margin is not in range [-1, 1].

Supported Platforms:: Ascend GPU CPU

Examples

>>> logits_x1 = Tensor(np.array([[0.3, 0.8], [0.4, 0.3]]), mindspore.float32)
>>> logits_x2 = Tensor(np.array([[0.4, 1.2], [-0.4, -0.9]]), mindspore.float32)
>>> labels = Tensor(np.array([1, -1]), mindspore.int32)
>>> cosine_embedding_loss = nn.CosineEmbeddingLoss()
>>> output = cosine_embedding_loss(logits_x1, logits_x2, labels)
>>> print(output)
0.0003425479

class tinyms.losses.SampledSoftmaxLoss(num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=True, seed=0, reduction='none')[source]¶

Computes the sampled softmax training loss. This operator can accelerate the training of the softmax classifier over a large number of classes. It is generally an underestimate of the full softmax loss.

Parameters

num_sampled (int) – The number of classes to randomly sample per batch.
num_classes (int) – The number of possible classes.
num_true (int) – The number of labels classes per training example. Default: 1.
sampled_values (Union[list, tuple]) – List or tuple of (sampled_candidates, true_expected_count, sampled_expected_count) returned by a *CandidateSampler function. Default to None, UniformCandidateSampler is applied.
remove_accidental_hits (bool) – Whether to remove “accidental hits” where a sampled class equals to one of the labels classes. Default: True.
seed (int) – Random seed for candidate sampling. Default: 0
reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. If “none”, do not perform reduction. Default: “none”.

Inputs:

weights (Tensor) - Tensor of shape \((C, dim)\).
bias (Tensor) - Tensor of shape \((C,)\). The class biases.
labels (Tensor) - Tensor of shape \((N, num\_true)\), type int64, int32. The labels classes.
logits (Tensor) - Tensor of shape \((N, dim)\). The forward activations of the input network.

Outputs:

Tensor or Scalar, if reduction is ‘none’, then output is a tensor with shape \((N,)\). Otherwise, the output is a scalar.

Raises

TypeError – If sampled_values is not a list or tuple.
TypeError – If dtype of labels is neither int32 not int64.
ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.
ValueError – If num_sampled or num_true is greater than num_classes.
ValueError – If length of sampled_values is not equal to 3.

Supported Platforms:: GPU

Examples

>>> mindspore.set_seed(1)
>>> loss = nn.SampledSoftmaxLoss(num_sampled=4, num_classes=7, num_true=1)
>>> weights = Tensor(np.random.randint(0, 9, [7, 10]), mindspore.float32)
>>> biases = Tensor(np.random.randint(0, 9, [7]), mindspore.float32)
>>> labels = Tensor([0, 1, 2])
>>> logits = Tensor(np.random.randint(0, 9, [3, 10]), mindspore.float32)
>>> output = loss(weights, biases, labels, logits)
>>> print(output)
[4.6051701e+01 1.4000047e+01 6.1989022e-06]

class tinyms.losses.DiceLoss(smooth=1e-05)[source]¶

The Dice coefficient is a set similarity loss. It is used to calculate the similarity between two samples. The value of the Dice coefficient is 1 when the segmentation result is the best and is 0 when the segmentation result is the worst. The Dice coefficient indicates the ratio of the area between two objects to the total area. The function is shown as follows:

\[dice = 1 - \frac{2 * (pred \bigcap true)}{pred \bigcup true}\]

Parameters: smooth (float) – A term added to the denominator to improve numerical stability. Should be greater than 0. Default: 1e-5.

Inputs:

logits (Tensor) - Tensor of shape \((N, *)\) where \(*\) means, any number of additional dimensions. The data type must be float16 or float32.
labels (Tensor) - Tensor of shape \((N, *)\), same shape as the logits. The data type must be float16 or float32.

Outputs:

Tensor, a tensor of shape with the per-example sampled Dice losses.

Raises

ValueError – If the dimension of logits is different from labels.
TypeError – If the type of logits or labels is not a tensor.

Supported Platforms:: Ascend GPU CPU

Examples

>>> loss = nn.DiceLoss(smooth=1e-5)
>>> logits = Tensor(np.array([[0.2, 0.5], [0.3, 0.1], [0.9, 0.6]]), mstype.float32)
>>> labels = Tensor(np.array([[0, 1], [1, 0], [0, 1]]), mstype.float32)
>>> output = loss(logits, labels)
>>> print(output)
0.38596618

class tinyms.losses.MultiClassDiceLoss(weights=None, ignore_indiex=None, activation='softmax')[source]¶

When there are multiple classifications, label is transformed into multiple binary classifications by one hot. For each channel section in the channel, it can be regarded as a binary classification problem, so it can be obtained through the binary loss of each category, and then the average value.

Parameters

weights (Union[Tensor, None]) – Tensor of shape \((num\_classes, dim)\). The weight shape[0] should be equal to labels shape[1].
ignore_indiex (Union[int, None]) – Class index to ignore.
activation (Union[str, Cell]) – Activate function applied to the output of the fully connected layer, eg. ‘ReLU’. Default: ‘softmax’. Choose from: [‘softmax’, ‘logsoftmax’, ‘relu’, ‘relu6’, ‘tanh’,’Sigmoid’]

Inputs:

logits (Tensor) - Tensor of shape \((N, C, *)\) where \(*\) means, any number of additional dimensions. The logits dimension should be greater than 1. The data type must be float16 or float32.
labels (Tensor) - Tensor of shape \((N, C, *)\), same shape as the logits. The labels dimension should be greater than 1. The data type must be float16 or float32.

Outputs:

Tensor, a tensor of shape with the per-example sampled MultiClass Dice Losses.

Raises

ValueError – If the shape of logits is different from labels.
TypeError – If the type of logits or labels is not a tensor.
ValueError – If the dimension of logits or labels is less than 2.
ValueError – If the weights.shape[0] is not equal to labels.shape[1].
ValueError – If weights is a tensor, but its dimension is not 2.

Supported Platforms:: Ascend GPU CPU

Examples

>>> loss = nn.MultiClassDiceLoss(weights=None, ignore_indiex=None, activation="softmax")
>>> logits = Tensor(np.array([[0.2, 0.5, 0.7], [0.3, 0.1, 0.5], [0.9, 0.6, 0.3]]), mstype.float32)
>>> labels = Tensor(np.array([[0, 1, 0], [1, 0, 0], [0, 0, 1]]), mstype.float32)
>>> output = loss(logits, labels)
>>> print(output)
0.54958105

class tinyms.losses.RMSELoss[source]¶

RMSELoss creates a criterion to measure the root mean square error between \(x\) and \(y\) element-wise, where \(x\) is the input and \(y\) is the labels.

For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the loss of \(x\) and \(y\) is given as:

\[loss = \sqrt{\frac{1}{N}\sum_{i=1}^{N}{(x_i-y_i)^2}}\]

Inputs:

logits (Tensor) - Tensor of shape \((N, *)\) where \(*\) means, any number of additional dimensions.
labels (Tensor) - Tensor of shape \((N, *)\), same shape as the logits in common cases. However, it supports the shape of logits is different from the shape of labels and they should be broadcasted to each other.

Outputs:

Tensor, weighted loss float tensor and its shape is zero.

Supported Platforms:

Ascend GPU CPU

Examples

>>> # Case 1: logits.shape = labels.shape = (3,)
>>> loss = nn.RMSELoss()
>>> logits = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> labels = Tensor(np.array([1, 2, 2]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
0.57735026
>>> # Case 2: logits.shape = (3,), labels.shape = (2, 3)
>>> loss = nn.RMSELoss()
>>> logits = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> labels = Tensor(np.array([[1, 1, 1], [1, 2, 2]]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
1.0

class tinyms.losses.MAELoss(reduction='mean')[source]¶

MAELoss creates a criterion to measure the average absolute error between \(x\) and \(y\) element-wise, where \(x\) is the input and \(y\) is the labels.

For simplicity, let \(x\) and \(y\) be 1-dimensional Tensor with length \(N\), the unreduced loss (i.e. with argument reduction set to ‘none’) of \(x\) and \(y\) is given as:

\[\ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad \text{with } l_n = \left| x_n - y_n \right|,\]

where \(N\) is the batch size. If reduction is not ‘none’, then:

\[\begin{split}\ell(x, y) = \begin{cases} \operatorname{mean}(L), & \text{if reduction} = \text{'mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{'sum'.} \end{cases}\end{split}\]

Parameters: reduction (str) – Type of reduction to be applied to loss. The optional values are “mean”, “sum”, and “none”. Default: “mean”.

Inputs:

logits (Tensor) - Tensor of shape \((M, *)\) where \(*\) means, any number of additional dimensions.
labels (Tensor) - Tensor of shape \((N, *)\), same shape as the logits in common cases. However, it supports the shape of logits is different from the shape of labels and they should be broadcasted to each other.

Outputs:

Tensor, weighted loss float tensor, the shape is zero if reduction is ‘mean’ or ‘sum’, while the shape of output is the broadcasted shape if reduction is ‘none’.

Raises: ValueError – If reduction is not one of ‘none’, ‘mean’, ‘sum’.

Supported Platforms:: Ascend GPU CPU

Examples

>>> # Case 1: logits.shape = labels.shape = (3,)
>>> loss = nn.MAELoss()
>>> logits = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> labels = Tensor(np.array([1, 2, 2]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
0.33333334
>>> # Case 2: logits.shape = (3,), labels.shape = (2, 3)
>>> loss = nn.MAELoss(reduction='none')
>>> logits = Tensor(np.array([1, 2, 3]), mindspore.float32)
>>> labels = Tensor(np.array([[1, 1, 1], [1, 2, 2]]), mindspore.float32)
>>> output = loss(logits, labels)
>>> print(output)
[[0. 1. 2.]
 [0. 0. 1.]]