NILG.AI

Meter tampering is a common threat to the business side of utility services and a public security threat, incurring uncontrolled tweaks that may increase the risk of accidents. So, fraud detection is core for any utility service company.

The presence of users that manipulate the network typically translates to an increased service price for the compliant users, since for sustainability of the services, remaining users absorb the additional load costs in addition to the inspection procedure costs associated with identifying such frauds.

This post showcases one of our previous AI projects on detecting meter tampering in water services. However, it can be used interchangeably in other types of services such as electricity, gas, and TV. The challenge we faced was identifying users with a high probability of meter tampering.

Quer discutir esta ideia mais a fundo?

Marcar uma reunião com Kelwin Fernandes

Meet Kelwin Saber mais

Modeling fraud detection for utility services

The first issue we faced was how reliable our labels were. While we had the outcomes from previous inspections (legal or fraud), inspections with a legal outcome may result -in some cases- from bribes. So, while you can trust positive fraud labels, negative cases are a mixture of actual legal cases and corruption itself. Another source of error on the legal observations are difficult-to-detect tampering, for example, in cases where the fraud was not directly on the meter but on bypassing the system at a different hidden point of the network.

We can model this learning task with multiple paradigms. 

1. In this challenge, most houses won’t have inspections. How can you use this data? Maybe you can consider them as negative for training, with the care of not introducing too much noise from hidden fraud, or you can use semi-supervised learning to regularize the model with this data (aqui).
2. Also, we discussed how to tackle this task with Positive Unlabeled learning in our previous blog post (aqui).
3. Since, in this case, we have access to the test set, Transductive Learning techniques may have a role to play here. If you don’t know what Transductive Learning is, take the time to read about it, chances are you may benefit from it (aqui).

Finally, This problem tends to be extremely unbalanced. Take a look at one of our previous papers on how to reformulate class imbalance as ranking tasks (aqui) which can be combined with any of the solutions mentioned above.

Curso

O Espectro do Machine Learning

Master many of these ML modeling techniques now!

Saber mais

The Data and Some Relevant Patterns for Fraud Detection

We trained our models combining three categories of data: contractual, consumption, and context data.

Contractual:

We included information about the:

1. Contract category: residential vs. industrial, dual-fee vs. flat-fee, contracted capacity, demographic data in the contract (e.g., house size, family size, etc.).
2. The victim itself – the potentially tampered meter: brand, model, capacity, technical specifications, etc. These features proved to be especially valuable when combined with context data (we will discuss this later).

While some features such as the ZIP code and the income capacity of the house may be correlated with fraud, we should take this with a grain of salt since we may be increasing bias and unfairness in the decision making (check this book).

Consumption:

Meter tampering often is associated with dramatic drops of consumption. Thus, we extracted features from aggregated consumption over time. Since consumption is seasonal and changes from house to house, it is essential to consider relative values instead of absolute ones. For example, you should observe the percentual change in usage between months instead of the absolute difference. We also normalized these features with respect to the global trend between any two months to compensate for seasonality.

Other fraud correlated patterns we found were consumption with low inter-month variability (i.e., the standard deviation of the consumption being too small) and “capped” expenditure (i.e., over the last year, the maximum value is repeated multiple times).

Contextual:

Context data proved to be the most important one. Meter tampering is like a virus and spreads among neighbors, especially among neighbors with the same model/brand of meter. Therefore, we extracted information such as the density of (recent) fraud within K meters (for multiple values of K) with and without stratification per meter model. Please pay attention to our emphasis on density and not a number. In this project, relativizing features to the local context of the individual and the region was critical to learn efficiently. If you don’t know how dense an area is (pretty common in countries with scarce open data), consider internal assets such as the number of contracts you have in the region and the density of your internal assets (e.g., pipes) as a proxy for density.

Reliable estimation of the model performance

When time and space are part of our learning system, we must pay attention to the way we split our data for performance estimation. Random splits will tend to “leak” information in training, giving overestimations of the model performance. In this case, we suggest splitting both temporally and geographically, as illustrated in the following graph.

Other variables to consider for splitting train and test might be the inspection rounds, inspection team, among others. The right split and its granularity (e.g., splitting between months/years, city/district/region) depend on business requirements and the way we intend to deploy our models.

How to use these predictions?

Predictions by themselves are not useful. We need to combine them with a decision process considering a KPI. In this case, we could use the additional information for fraud detection to build cost-effective inspection schedules and routes, finding the right trade-off between the expected benefit obtained from inspecting a house versus the cost of such inspection. We will cover this topic on a different blog post.

Course, Templates

Data Ignite

Learn how to create valuable AI solutions from predictions to actions.

Saber mais

Other use cases in the industry beyond fraud detection

We discussed here how to detect fraud detection of water meter. However, the utility industry is a vast field for the application of AI, from determining regions with a high potential value of prospect acquisition, churn prediction, anomaly detection in the network, among others. If you want to explore any of these ideas, just message us!