Loss¶
Loss module includes a set of training loss implmentations. Some are converted from C++ implementation, and the rest are implemented directly using python Tensor.
Example usage:
from singa import tensor
from singa import loss
x = tensor.Tensor((3, 5))
x.uniform(0, 1) # randomly genearte the prediction activation
y = tensor.from_numpy(np.array([0, 1, 3], dtype=np.int)) # set the truth
f = loss.SoftmaxCrossEntropy()
l = f.forward(True, x, y) # l is tensor with 3 loss values
g = f.backward() # g is a tensor containing all gradients of x w.r.t l

class
singa.loss.
Loss
¶ Bases:
object
Base loss class.
Subclasses that wrap the C++ loss classes can use the inherited foward, backward, and evaluate functions of this base class. Other subclasses need to override these functions

backward
()¶ Returns: the grad of x w.r.t. the loss

evaluate
(flag, x, y)¶ Parameters: Returns: the averaged loss for all samples in x.

forward
(flag, x, y)¶ Compute the loss values.
Parameters: Returns: a tensor of floats for the loss values, one per sample


class
singa.loss.
SigmoidCrossEntropy
(epsilon=1e08)¶ Bases:
singa.loss.Loss
This loss evaluates the crossentropy loss between the prediction and the truth values with the prediction probability generated from Sigmoid.

backward
()¶ Compute the gradient of loss w.r.t to x.
Returns: dx = pi  yi.

evaluate
(flag, x, y)¶ Compuate the averaged error.
Returns: a float value as the averaged error


class
singa.loss.
SoftmaxCrossEntropy
¶ Bases:
singa.loss.Loss
This loss function is a combination of SoftMax and CrossEntropy loss.
It converts the inputs via SoftMax function and then computes the crossentropy loss against the ground truth values.
For each sample, the ground truth could be a integer as the label index; or a binary array, indicating the label distribution. The ground truth tensor thus could be a 1d or 2d tensor. The data/feature tensor could 1d (for a single sample) or 2d for a batch of samples.

class
singa.loss.
SquaredError
¶ Bases:
singa.loss.Loss
This loss evaluates the squared error between the prediction and the truth values.
It is implemented using Python Tensor operations.

backward
()¶ Compute the gradient of x w.r.t the error.
Returns: x  y

evaluate
(flag, x, y)¶ Compuate the averaged error.
Returns: a float value as the averaged error

forward
(flag, x, y)¶ Compute the error as 0.5 * xy^2.
Parameters: Returns: a Tensor with one error value per sample
