By Dave DeFusco
In a world powered by sensors, smartwatches and the Internet of Things (IoT), enormous streams of data are constantly being collected about our health, our environment and our devices. But sometimes that data contains outliers—numbers that don’t fit the normal pattern. Detecting those outliers early can prevent major problems, from identifying a failing machine part to spotting the first signs of a heart attack.
The challenge, though, is that most tools for finding these unusual data points, especially those using machine learning or deep learning, are heavy, complex and power-hungry. They work great on big computers but struggle to run on the smaller, low-power devices we carry or install in our homes.
That’s what inspired Dr. David Li, director of the Katz School’s M.S. in Data Analytics and Visualization, to design an algorithm, called IDOS (Interpolated Density for Outlier Score), which is fast, efficient and doesn’t require much computing power.
“We wanted to create a model that could think smart but act light,” said Dr. Li. “IDOS is designed for real-world environments where every bit of power and memory matters, like wearable health monitors, embedded sensors and other IoT devices.”
Dr. Li and his co-authors—Yu Wang, a student in the Katz School’s M.S. in Artificial Intelligence, and Angela Li, a researcher at Stony Brook University—presented their work in March at the 59th Annual Conference on Information Science and Systems at Johns Hopkins University.
An outlier is simply a data point that doesn’t behave like the rest. In healthcare, for instance, if your heart rate or blood sugar suddenly spikes far beyond your normal range, that could be an early warning sign of trouble. Outlier detection algorithms act as silent sentinels that are always watching for those warning signals that might otherwise go unnoticed.
Traditional methods fall into two extremes: simple rule-based systems that might flag a reading if it crosses a fixed limit—useful but rigid, and sophisticated deep-learning models that can detect complex patterns but demand massive amounts of data and computational power.
“Most algorithms require tuning lots of parameters or running on high-performance servers,” said Wang. “That makes them difficult to deploy on devices that have limited battery life or memory.”
IDOS takes a fresh approach. Instead of assuming data follows a specific shape or pattern, it adapts to the data itself. It uses what’s called a nonparametric method, meaning it doesn’t rely on fixed assumptions about how data is distributed.
At the heart of IDOS is something called the Empirical Cumulative Distribution Function, or CDF, which measures how data values are spread out, from smallest to largest, and helps the algorithm see what normal looks like. IDOS then uses interpolation, a mathematical way of filling in gaps smoothly between data points, to judge how far a new data point strays from normal.
Because it works directly with the data rather than a pre-set model, IDOS doesn’t need extensive parameter tuning or big datasets to learn from. “We wanted it to work right out of the box,” said Angela Li. “You don’t need to be a data scientist to use it or to spend hours adjusting settings. It just adapts.”
To see IDOS in action, the team tested it on a health dataset involving patients at risk for heart attacks. They analyzed features like age, heart rate, blood pressure, blood sugar and cardiac enzyme levels—key indicators of cardiovascular health.
IDOS successfully flagged abnormal readings that signaled potential heart problems, using a small fraction of the computing power required by other methods. It also matched or outperformed more complex algorithms in detecting anomalies.
Even better, it could process data continuously, which is ideal for wearable health devices. “Imagine your smartwatch quietly tracking your vitals and alerting you instantly when something looks off,” said Wang. “IDOS makes that kind of real-time monitoring feasible, even on limited hardware.”
Unlike many machine-learning algorithms that must be retrained for every new situation, IDOS can be easily deployed across different devices and scenarios, from health monitoring to industrial sensors and even cybersecurity systems.
“It’s a simple idea with wide-reaching potential,” said Dr. Li. “Whether it’s detecting a failing component in an aircraft, an unauthorized login in a network or an abnormal heartbeat in a fitness tracker, IDOS brings powerful anomaly detection to the edge, where data is generated.”
