Machine Learning Model Evaluation Metrics: Precision, Recall & mAP

When deploying computer vision and machine learning models in real-world environments (such as in [[Live With Paws — AI Wildlife Intrusion Detection System]]), raw accuracy is rarely enough. Evaluating model performance requires understanding Precision, Recall, F1-Score, and Mean Average Precision (mAP).

📊 The Confusion Matrix

In binary and multi-class classification, prediction outcomes fall into four quadrants:

| | Predicted Positive | Predicted Negative | | :--- | :--- | :--- | | Actual Positive | True Positive (TP) | False Negative (FN) | | Actual Negative | False Positive (FP) | True Negative (TN) |

🧮 Core Formulae & Metric Trade-Offs

1. Precision

Precision measures how many of the positive predictions were actually correct: $\text{Precision} = \frac{TP}{TP + FP}$ High precision minimizes false alarms (False Positives).

2. Recall (Sensitivity)

Recall measures how many of the actual positive cases were correctly identified by the model: $\text{Recall} = \frac{TP}{TP + FN}$ High recall ensures critical events (False Negatives) are not missed.

3. F1-Score

The harmonic mean of Precision and Recall: $F1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$

🎯 Object Detection Evaluation: Intersection over Union (IoU) & mAP

In object detection systems like YOLOv8:

1. Intersection over Union (IoU): Measures the spatial overlap between predicted bounding box $B_p$ and ground truth bounding box $B_{gt}$: $\text{IoU} = \frac{\text{Area}(B_p \cap B_{gt})}{\text{Area}(B_p \cup B_{gt})}$
2. Mean Average Precision (mAP): Evaluates Average Precision (AP) across detection confidence thresholds and object categories (e.g. mAP@50 and mAP@50-95).

💻 Python Example using Scikit-Learn

from sklearn.metrics import classification_report, confusion_matrix
import numpy as np

# Ground truth vs predicted labels
y_true = np.array([1, 0, 1, 1, 0, 1, 0, 0, 1, 0])
y_pred = np.array([1, 0, 1, 0, 0, 1, 1, 0, 1, 0])

# Confusion matrix and evaluation metrics
print("Confusion Matrix:")
print(confusion_matrix(y_true, y_pred))

print("\nClassification Report:")
print(classification_report(y_true, y_pred, target_names=["No Intrusion", "Intrusion"]))

🔗 Related Notes & Projects

• Applied Case Study: [[Live With Paws — AI Wildlife Intrusion Detection System]]
• Data Manipulation: [[Pandas Data Analysis & Tabular Mechanics]]
• Visual Analytics: [[Data Visualization with Matplotlib & Pandas: Fundamentals & Practice]]