Technical Anomalies in Water-Quality Data From In-Situ Sensors - What, Why, and How?

Abstract

Anomalies in water quality data due to technical errors from in situ sensors can reduce data quality and have a direct impact on inference drawn from subsequent data analysis. To identify these anomalies, we need a statistical procedure to distinguish an anomaly due to a technical error from other anomalies, and from regular data. Anomaly detection algorithms are highly influenced by the way we define a statistical anomaly. In this work we define an anomaly as an observation that has an unexpectedly low conditional probability distribution. Different types of anomalies can be detected by varying what we condition on. In this talk, I will first discuss what are the different types of technical anomalies that can present in water-quality sensor data positioned at different geographic sites of a river network. Second, I will explain why it is important to use various types of conditioning information such as contemporaneous downstream observations, lagged downstream observations, and upstream observations at the time the conditional correlation is maximised, to improve technical anomaly detection in river networks. Third, I will discuss the differences between our previously proposed oddwater algorithm and this new approach for detecting anomalies in water-quality variables, and how we can expect the extended framework to allow us to deal with a wider range of technical anomaly types in water-quality sensor data.

Date
Nov 28, 2021 12:00 AM — 12:00 AM
Event
ARCLP Workshop
Avatar
Priyanga Dilini Talagala
PhD in Statistics

My research interests include Computational Statistics, Anomaly Detection, Time Series Analysis and Machine Learning.