Machine Learning · AI216

ROC Curve and AUC Explained

Machine Learning Study guides

The ROC curve plots the true positive rate (recall) against the false positive rate at every decision threshold; the area under it (AUC) summarises classifier quality in one number.

A classifier like SGDClassifier computes a score with a decision function. If the score is greater than a threshold the instance is assigned to the positive class, otherwise to the negative class. Moving the threshold changes the numbers of true and false positives. The ROC (receiver operating characteristic) curve shows that whole trade-off: the y-axis is TPR (recall) and the x-axis is FPR = FP / (FP + TN), which equals 1 - specificity.

A perfect classifier has ROC AUC = 1. A purely random classifier gives the diagonal line and an AUC of 0.5. The higher the recall, the more false positives the classifier produces, so a good curve hugs the top-left corner. Youden index = sensitivity + specificity - 1 and marks the best threshold point.

Rule of thumb: prefer the precision/recall curve when the positive class is rare or when you care more about false positives than false negatives; otherwise ROC is a good general view. ROC is a tool for binary classifiers.

Key points

  • ROC: TPR (y) versus FPR (x) across thresholds.
  • AUC = 1 perfect, 0.5 random.
  • Higher recall comes with more false positives.
  • Youden index = sensitivity + specificity - 1.
  • Use the PR curve for rare positive classes.

Worked example

If at one threshold a classifier finds 90 percent of positives (TPR 0.90) and wrongly flags 20 percent of negatives (FPR 0.20), that is one point at (0.20, 0.90). Sweeping the threshold draws the whole curve.

Common questions

What is the ROC AUC of a random classifier?

0.5.

What does ROC stand for?

Receiver operating characteristic.

Practice questions on this topic

Try these, then open the app to check your answers with explanations.

  1. In the basic Scikit-Learn classifier API as taught in the course, Scikit-Learn lets you set the threshold directly
    • A. false
    • B. true
  2. ....................evaluates the discriminative power of the test
    • A. YI
    • B. MCC
    • C. DSC
    • D. none of them
  3. If that score is greater than a threshold, it assigns the instance to the ............
    • A. positive class
    • B. negative class
    • C. both of them
    • D. none of them
  4. precision recall curve() computes..............for all possible thresholds.
    • A. precision and recall
    • B. precision
    • C. recall
    • D. F-measure
  5. What is the ROC AUC of a purely random classifier
    • A. 0.5
    • B. 1
    • C. 0
    • D. none of them
  6. If that threshold is greater than a score, it assigns the instance to the ............
    • A. negative class
    • B. positive class
    • C. both of them
    • D. none of them