Confusion Matrix, Precision, Recall and F1 Score
A confusion matrix counts correct and wrong predictions per class; precision, recall and F1 score are built from its true and false positives and negatives.
In a binary confusion matrix each row is an actual class and each column is a predicted class (in Scikit-Learn). The four cells are true positives (TP), false positives (FP), true negatives (TN) and false negatives (FN). A perfect classifier has only true positives and true negatives, so its matrix is non-zero only on the main diagonal.
Precision = TP / (TP + FP): of everything predicted positive, how much was really positive. Recall (also sensitivity or true positive rate) = TP / (TP + FN): of all actual positives, how many were found. Specificity (true negative rate) = TN / (TN + FP). Accuracy = (TP + TN) / total.
F1 score is the harmonic mean of precision and recall: F1 = 2 x precision x recall / (precision + recall). Because it is a harmonic mean it is only high if both are high. Precision and recall pull against each other: raising the decision threshold increases precision and lowers recall, and vice versa. This is the precision/recall trade-off.
Key points
- Precision = TP/(TP+FP), recall = TP/(TP+FN).
- F1 is the harmonic mean of precision and recall.
- Raising precision usually lowers recall.
- Accuracy can mislead on skewed classes; the confusion matrix is a better evaluation.
- Specificity = TN/(TN+FP); 1 - accuracy is the error rate.
Worked example
A cancer test gives TP=40, FP=10, FN=20, TN=930. Precision = 40/50 = 0.80. Recall = 40/60 = 0.67. F1 = 2 x 0.80 x 0.67 / 1.47 = 0.73. Accuracy = 970/1000 = 0.97, which hides the 20 missed patients.
Common questions
What is the difference between precision and recall?
Precision asks how many predicted positives are right. Recall asks how many real positives were found.
Why use harmonic mean for F1?
It punishes a big gap between precision and recall; a classifier gets a high F1 only if both are high.
Practice questions on this topic
Try these, then open the app to check your answers with explanations.
- ................is the inverse precision.
- A. Negative Predictive Value
- B. Positive Prediction Value
- C. True Negative Rate
- D. none of them
- the combination of precision and recall into a single metric called confusion matrix
- A. false
- B. true
- The higher the recall (TPR), the more false positives (FPR) the classifier produces.
- A. true
- B. false
- Increasing precision reduces recall, and vice versa. This is called the precision/recall trade-off
- A. true
- B. false
- What is the purpose of a confusion matrix?
- A. To evaluate the performance of a classification algorithm
- B. To identify the most important features in a dataset
- C. To select the right machine learning algorithm for a given problem
- D. To tune the hyperparameters of a machine learning algorithm
- the confusion matrix of perfect classifier would have ............. values only on its main diagonal
- A. nonzero
- B. zero
- C. ones
- D. none of them
Related topics
- ROC Curve and AUC Explained
- K-Fold and Stratified K-Fold Cross-Validation
- Multiclass Classification: One-versus-One and One-versus-Rest
- MNIST Dataset and SGDClassifier Explained
- What is Machine Learning?
- Supervised vs Unsupervised vs Semi-supervised vs Reinforcement Learning
- All topics: Machine Learning
BASMAGLY is an independent student project, not an official Delta University site. Check lectures and your instructor for the final word.