Applying Zipf's law in fraud detection

Investigating aggregate risk scores

Within Splunk products, we often assign risk scores to fraudulent patterns of behavior. If an aggregate risk score goes over a predetermined threshold, the result might be that this entity has engaged in fraudulent activity. Fortunately, most aggregate risk scores do not go over the threshold, which means only a partial set of entities need to be investigated.

Here’s a distribution graph. Notice the descending pattern in this distribution follows the Zipf’s law ideas.

In this case, the investigations will be for the highest occurring aggregate risk scores rather than the lowest ones, which are probably benign in a system that accurately measures risks.

Finding unusual wire transfers

A wire transfer might occur from one bank to another bank in another country. The pattern of distribution of frequencies for the count of countries appearing as the recipient country will be descending and follow a curve similar to Zipf’s law. There might be some countries that rarely appear on any given day as the recipient country. 

Most wire transfers will occur more than once to a country where wire transfer is expected due to the nature of the business and connected entities. If a wire transfer only occurs once for a country in a day, it might be suspicious. This does not mean it is fraud, but it does add to the risk.

In the Splunk platform we can use the rare command to see this.

index=transactions sourcetype=wire_transfer
|``` transformation, where clauses, etc
|rare destCountry
|sort - count

In this example, we use the rare command to count the frequency of the rarest countries that are the destination countries of a wire transfer. Because rare returns count sizes least to highest, in order to be consistent with our other charts, we sort the results from highest to lowest in the chart. This doesn’t exactly follow Zipf’s law, but it does show a reduction curve that follows the law in spirit.

What is missing here is context for the transaction such as the account name, the amount, etc. Let’s add that into the search.

index=transations sourcetype=wire_transfer
|``` comment: transformation, where clauses, etc
|stats count, list(_time) AS Time, values(customer) AS Customer, values(FromAccount) AS FromAccount, values(ToAccount) AS ToAccount, values(action) AS action, values(amount) AS amount BY destCountry 
|where count=1
|eval RiskScore_DestCountry=30

In this example, we are using the stats command to save context for the transaction so it can be used to feed an alert or collect into a risk index. We no longer need to use rare because counting the destination countries’ frequency can get us the rarest count, and in this case the rarest count is exactly one. In the end, an arbitrary risk score is added to the results of the search so that it can be stored in a risk index for further aggregation of risk scores against the same entity.

This additional risk score might be the one that tips the aggregate risk score for the entity over a predetermined threshold due to other risk scores for the entity, which then means it is probably fraud.