Anomaly Detection
A detector that never fires scores 99.5% accuracy, a ROC of 0.9468 hides an alert queue that is 64% false, distance from the mean scores below chance when the anomalies sit at the centre, and twenty anomalies that group together hide each other from the method built to find them.
Sign in to take quizzes, earn XP, and unlock stages as you reach 90% mastery.
What Accuracy Hides
25 min · 100 XPAt a 0.5% base rate the do-nothing detector wins on accuracy, ROC reports 0.9468 for a model whose top hundred alerts are 64% false, and precision@k is the only number that describes the queue somebody has to work through.
Open lesson →Sign in to take the 3-question quiz.
Distance, Density and Isolation
30 min · 120 XPFour detectors on data with two kinds of anomaly: distance from the centre scores below chance, an axis-parallel splitter never sees the gap, and the local method is defeated by anomalies that group together.
Open lesson →Sign in to take the 3-question quiz.
Thresholds, Alerts and Drift
28 min · 110 XPA threshold set from the cost of a miss against the cost of a false alarm rather than from a quantile, and the measurement showing that a fixed threshold reprices the alert queue whenever the anomaly rate moves.
Open lesson →Sign in to take the 3-question quiz.