DIMACS Monitoring Message Streams Working Group: Spring 2004

This Spring our seminar series will meet on Friday afternoons and everyone is welcome.

Working Group: Spring 2004
Date Speaker Topic
March 5th, 3:30pm Diane Lambert, Bell Labs Dynamic thresholding for network monitoring
April 9th, 2:30pm David Holmes, College of New Jersey TBA
May 7th, 11am Tony Jebara, Columbia University TBA
May 7th, 3:30pm Mark Goldberg, RPI TBA

Dynamic Thresholding for Network Monitoring
Diane Lambert, Bell Labs, Lucent Technologies

Streams of counts on traffic volume and processing errors are continually compared to thresholds to detect network degradation. Thresholds are usually set by hand, which is tedious and error-prone, or by relying on normality assumptions, which gives unacceptably large false alarm rates. This talk will describe a simple, statistically principled approach to automated thresholding that starts from the statistical properties of the counts and then considers algorithms that meet the computing constraints. Longterm trends, cyclical patterns, outliers, and missing values are automatically accommodated in the thresholds. Just as importantly, events (periods of anomalous behavior) can be prioritized in terms of severity and duration, even though count distributions differ over time. Performance on real and simulated data will be described. (Joint work with Chuanhai Liu.)