Can I use AUC as loss function?

Can I use AUC as loss function?

For binary classification, Receiver Operating Characteristic (ROC) curve incorporates different evaluation metrics. The Area Under ROC Curve (AUC) is a widespread metric, especially in Medical Science [1]. used AUC as a loss function and demonstrated AUC-based training lead to better generalization [6].

What is AUC maximization?

Deep AUC Maximization (DAM) is a new paradigm for learning a deep neural network by maximizing the AUC score of the model on a dataset. First, we propose a new margin-based min-max surrogate loss function for the AUC score (named as AUC min-max-margin loss or simply AUC margin loss for short).

What is AUC loss?

AUC is a reciprocal of the ranking loss, which is similar to the 0 − 1 loss, in the sense that it measures the percentage of pairs of data samples, one from the negative class and one from the positive class, such that the classifier assigns a larger label to the negative sample than to the positive one.

What is log loss and ROC AUC?

AUC (ROC) improves when the order of the predictions becomes more correct. And logloss deteriorates when there are more confident false predictions.

What is the interpretation of an ROC area under the curve as an integral?

The Area Under the Curve (AUC) is the measure of the ability of a classifier to distinguish between classes and is used as a summary of the ROC curve. The higher the AUC, the better the performance of the model at distinguishing between the positive and negative classes.

What is good Logloss?

The logloss is simply L(pi)=−log(pi) where p is simply the probability attributed to the real class. So L(p)=0 is good, we attributed the probability 1 to the right class, while L(p)=+∞ is bad, because we attributed the probability 0 to the actual class.

What is log loss and how it helps to improve performance?

Log-loss is an appropriate performance measure when you’re model output is the probability of a binary outcome. The log-loss measure considers confidence of the prediction when assessing how to penalize incorrect classification.

Is it possible to optimise ROC AUC directly using approximation?

Tflearn provides the option of optimising ROC AUC directly using an approximation suggested in this paper. If this is something people think it would be worth adding, I would be happy to give it a go. Sorry, something went wrong. you may try pairwise ranking loss.

Why is AUC not a smooth function of probabilities?

The AUC in the equation above is not a smooth function of probabilities, because it is constructed using step functions. We can try to smooth it out, to make sure the function is always differentiable, so that we can use the conventional optimization algorithms.

What is the relationship between AUC and log loss?

As you mention, AUC is a rank statistic (i.e. scale invariant) & log loss is a calibration statistic. One may trivially construct a model which has the same AUC but fails to minimize log loss w.r.t. some other model by scaling the predicted values. Consider: So, we cannot say that a model maximizing AUC means minimized log loss.

How differentiable is a AUC?

AUC is not differentiable, but it’s equivalent to the expected probability that a classifier will correctly rank a random positive and random negative example. If your classifier outputs probabilities (or something you can treat as probabilities), you could try this as a proxy loss: