In this post, we will learn how to detect fraud when our data is not labeled with ground truths for testing purposes. Previously, we determined the number of clusters that are appropriate for our data. Now, we will take this knowledge and identify potential examples of fraud.

Libraries

We will start with the necessary libraries and the loading of our data.

from sklearn.cluster import KMeans
from scipy.spatial.distance import cdist
from sklearn.model_selection import train_test_split
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix, classification_report, roc_auc_score
from sklearn.preprocessing import MinMaxScaler
df = pd.read_csv("C:/Users/dthom/Documents/python/fraud/chapter_3/chapter_3/banksim_adj.csv")

The KMeans algorithm will be used to label our data as belonging to specific clusters. The cdist() function will be used to calculate the distance between individual data points and the cluster centroids. The train_test_split() function will be used to divide our data into training and testing sets. pandas and numpy will be used for data cleaning. matplotlib will be use to create a visualization. The confusion_matrix(), classification_report, and roc_auc_score functions will be used for model assessment. Lastly, the MinMaxScaler() function will be used to scale the data.

Data Preparation

In the code below, we scale the X values and create our training and testing sets. Notice how we leave out column 3 in the X values, as this column is our y values. The y values are actually the labels of fraud. What we are doing here is assuming we don’t have the values. We will then compare what we find in our analysis using Kmeans to the y values.

X = df.iloc[:, [1, 2] + list(range(4, 19))]  
y = df.iloc[:, 3]    

# Take the float values of df for X
X = df.values.astype(float)

# Define the scaler and apply to the data
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

# Split the data into training and test set
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.3, random_state=0)

We are not scaling our y values because they are not used in creating our clusters.

Model Development

We will now create our clusters. We already know that using three clusters is appropriate, so we use the KMeans() function to set this value along with the seed number of 42. We then take this information and predict cluster values for the test data. Last, we put the cluster centers into their own object and display the values to show what is happening with the data.

# Define K-means model 
kmeans = KMeans(n_clusters=3, random_state=42).fit(X_train)

# Obtain predictions
X_test_clusters = kmeans.predict(X_test)
X_test_clusters_centers = kmeans.cluster_centers_
print(X_test_clusters_centers)

[[ 4.86803753e-01  4.94050179e-01  1.11442274e-01  1.49186219e-16
   1.60982339e-15 -6.93889390e-18  1.73472348e-18 -6.59194921e-17
   4.85722573e-17  1.73472348e-16 -1.86482774e-17  7.80625564e-18
  -2.60208521e-17 -2.81892565e-18 -1.40946282e-18  4.85722573e-17
   2.42861287e-17  1.00000000e+00  1.21430643e-17]
 [ 4.86841962e-01  5.03923667e-01  1.12597701e-01  8.32667268e-17
   1.00000000e+00  2.25514052e-17 -3.46944695e-18 -2.77555756e-17
   1.17961196e-16  1.31838984e-16 -1.04083409e-17  1.21430643e-17
   1.04083409e-17 -2.38524478e-18 -1.19262239e-18  3.98986399e-17
   1.99493200e-17  1.00000000e+00  9.97465999e-18]
 [ 5.69661624e-01  4.93834527e-01  3.60411064e-01  1.69451074e-01
   4.03341289e-01  7.51789976e-02  1.43198091e-02  7.39856802e-02
   2.39856802e-01  1.96897375e-01  2.26730310e-02  2.62529833e-02
   6.32458234e-02  9.54653938e-03  4.77326969e-03  6.68257757e-02
   3.34128878e-02  1.66533454e-15  1.67064439e-02]]

We will now calculate the distance each data point is from its assigned cluster. The np.linalg.norm() function computes the norm (magnitude) of a vector or matrix in NumPy, and we do this for every data point.

#calculate distance from cluster centroid
dist = [np.linalg.norm(x-y) for x, y in zip(X_test, X_test_clusters_centers[X_test_clusters])]
print(dist)

[0.3010076776055849, 1.124798110868828, 0.6818089230629063, 0.7301151147782995, 1.0838984579838724, 0.20656858215304805, 0.4606011452724887, 0.09018296562281172, 0.4918133960385591]

Now that we know the distances, we look for extreme values from the centroids. We convert our dists into percentiles and calculate the values that are in the 95th percentile or higher

# Create fraud predictions based on outliers on clusters 
km_y_pred = np.array(dist)
km_y_pred[dist >= np.percentile(dist, 95)] = 1
km_y_pred[dist < np.percentile(dist, 95)] = 0

Values that are in the 95th percentile or above are labeled as 1, while all other values are labeled as 0. The values labelled as 1 could potentially be instances of fraud. In a real situation with unlabeled data, this is where your analysis would end, and you would turn the results over to a subject expert who would determine which examples are fraudulent.

Model Assessment

Now we will see how well our model performs under the assumption of the data points we identified as fraudulent. The confusion matrix and ROC Score are below.

# Create a confusion matrix
km_cm = confusion_matrix(y_test, km_y_pred)

# Obtain the ROC score
print("ROC Score {}"  .format(roc_auc_score(y_test, km_y_pred)))
print(classification_report(y_test, km_y_pred)) # false positives!

Roc Score 0.9703717698082832
              precision    recall  f1-score   support

           0       1.00      0.98      0.99      2099
           1       0.52      0.97      0.67        58

    accuracy                           0.97      2157
   macro avg       0.76      0.97      0.83      2157
weighted avg       0.99      0.97      0.98      2157

The results indicate a strong model overall, as the ROC Score is close to 1 (0.97), but the model struggles with false positives. We know this from the precision being 0.52. In the code below, we create a visual of the confusion matrix

# Plot the confusion matrix in a figure to visualize results 
km_cm= confusion_matrix(y_test, km_y_pred)
from matplotlib.colors import ListedColormap
import seaborn as sns
ax= plt.subplots(figsize=(5,5))
sns.set(font_scale=1.4)
with sns.axes_style('white'):
    sns.heatmap(km_cm, cbar=False, square=True,annot=True,fmt='g',
cmap=ListedColormap(['gray']),linewidths=2.5)
plt.show()

You can see the false positives in the upper right-hand corner. The context determines if this is a problem.

Conclusion

The example above illustrates one method for identifying fraudulent examples in a dataset. This was a multi-step process that involved creating clusters and determining which examples were considered outliers in comparison to the cluster centroids.