Training and Inference Steps

Training Step

We extract various attributes—also known as features—using the metadata from the Salesforce application logs. We use metadata about report generation and surrounding activities over a period of 90 days. The actual list of features changes as the model improves.

Using these features, we build a model of the user's typical report generation activity. This step is called model training. We use the trained model to detect anomalies in the second step.

Inference (or Detection) Step

During the detection step, we look at every report generation activity for every user and extract the same set of features used to train the model. We then compare features against the model of the user's typical behavior and determine if the activity under consideration is sufficiently different.

Anomaly Score

We assign a numerical anomaly score to every report generation activity based on how different the activity is compared to the user’s typical activity. The anomaly score is always a number from 0 through 100, and is often expressed as a percentage. A low anomaly score indicates that the user's report generation activity is similar to the user's typical activity. A high anomaly score indicates that the user's report generation activity is different from the user's typical activity.

Critical Threshold

Every report generation event is assigned an anomaly score, but not all generation events are anomalies. We use a threshold to determine which report generation events are sufficiently different from a user’s typical activity. Any event with an anomaly score above the critical threshold is considered an anomaly.