Confusion Matrix Simulator

How Confusion Matrix Simulator works

A confusion matrix cross-tabulates what a classifier predicted against what was actually true, giving four counts: true positives, false positives, true negatives and false negatives. Every headline metric is a ratio of those four numbers. Accuracy, (TP + TN) / total, is the fraction called correctly — and is misleading whenever the classes are imbalanced, since always predicting the majority class scores well. Precision, TP / (TP + FP), asks how many of the items flagged positive really were; recall, TP / (TP + FN), asks how many of the real positives were caught. The two trade off against each other through the decision threshold: lowering it flags more items, raising recall while precision falls, and raising it does the reverse. F1, the harmonic mean 2PR / (P + R), summarises the pair and stays low unless both are decent. Which one matters is a domain question — a spam filter protects precision because a lost email is costly, a disease screen protects recall because a missed case is worse.

Pseudocode

confusionMatrix(scores, labels, threshold):
    TP = FP = TN = FN = 0
    for each (score, actual) in zip(scores, labels):
        predicted = (score >= threshold) ? POSITIVE : NEGATIVE
        if   predicted == POSITIVE and actual == POSITIVE: TP += 1
        elif predicted == POSITIVE and actual == NEGATIVE: FP += 1
        elif predicted == NEGATIVE and actual == NEGATIVE: TN += 1
        else:                                              FN += 1
    return (TP, FP, TN, FN)

metrics(TP, FP, TN, FN):
    accuracy    = (TP + TN) / (TP + FP + TN + FN)
    precision   = TP / (TP + FP)         // guard: 0 predicted positives
    recall      = TP / (TP + FN)         // a.k.a. sensitivity, TPR
    specificity = TN / (TN + FP)         // TNR
    f1          = 2 * precision * recall / (precision + recall)
    return { accuracy, precision, recall, specificity, f1 }

// Sweeping the threshold from high to low traces the ROC curve:
//   point = (FPR = FP / (FP + TN),  TPR = recall)

Time and space complexity

Time

O(n) to build the matrix, O(n log n) to sweep every threshold for a ROC curve

Space

O(1) for the four counts; O(c²) for a c-class matrix

The matrix is one pass over the predictions. A ROC or precision-recall curve needs the scores sorted once, after which each threshold step updates the counts incrementally.

Worked example

100 patients screened, 20 actually have the disease. At the current threshold the model flags 30 people, of whom 18 are genuinely ill.

1. TPFlagged and ill = 18.
2. FPFlagged but healthy = 30 − 18 = 12.
3. FNIll but not flagged = 20 − 18 = 2.
4. TNHealthy and not flagged = 100 − 18 − 12 − 2 = 68.
5. Accuracy(18 + 68) / 100 = 0.86.
6. Precision18 / (18 + 12) = 0.60 — most flags are false alarms.
7. Recall18 / (18 + 2) = 0.90 — nearly every case caught.
8. F12(0.60)(0.90) / (0.60 + 0.90) = 1.08 / 1.50 = 0.72.

Accuracy of 0.86 flatters the model: predicting "healthy" for everyone would score 0.80 with recall 0. The precision/recall split shows what it is really doing — catching 90% of cases at the cost of twelve false alarms, a sensible trade for a screening test.

Common exam mistakes

• Reading accuracy on imbalanced data. With 1% positives, a model that never predicts positive is 99% accurate and completely useless.
• Swapping precision and recall. Precision divides by what you predicted positive (TP + FP); recall divides by what actually was positive (TP + FN).
• Confusing a false positive with a false negative. The word names the prediction, not the truth — a false positive was predicted positive and was wrong.
• Using the arithmetic mean of precision and recall instead of F1. The harmonic mean is deliberately unforgiving: 1.0 precision with 0.0 recall gives F1 = 0, not 0.5.
• Comparing models at different thresholds. Precision and recall are threshold-dependent; compare at a fixed threshold, or compare curves such as ROC-AUC.

Frequently asked questions

Related simulators