You are here:
Algorithms and Loss Functions
Predictive models analyze historical data to discover patterns and relationships that predict future outcomes. Model Builder provides algorithms for binary, multiclass, and regression models with configurable training settings. When you train a predictive model with a numeric outcome, loss functions measure how far predicted values are from actual values. During training, the model minimizes this loss to improve overall prediction accuracy. The selected loss function determines the types of prediction errors that the model prioritizes.These settings enable you to generate predictions that drive business decisions and surface customer insights.
Supported Algorithms
| Algorithm | Description |
|---|---|
| AdaBoost | AdaBoost is an ensemble algorithm that trains models in sequence and focuses on previously misclassified data to improve predictions. |
| CatBoost | CatBoost is a gradient boosting algorithm that efficiently handles categorical variables. It delivers accurate predictions with minimal data preprocessing. |
| Decision Tree | Decision Tree is a supervised algorithm that builds a tree of decision rules. It makes predictions easy to interpret and visualize. |
| GLM | Default. Generalized Linear Model (GLM) is an equation-based algorithm that typically completes quickly. It works best when the relationship between the variables and the outcome is relatively simple. Only GLM can produce explanations based on interactions between variables, such as the combined impact of region and month. |
| GBM | Gradient Boosting Machine (GBM) is a tree-based algorithm in which decision trees are built sequentially to better fit the data. Compared with GLM, it handles complex data, such as different relationships between variables and varied data distributions, better. |
| LightGBM | Light Gradient Boosting Machine (LightGBM) is a fast, scalable gradient boosting algorithm that builds decision trees to deliver highly accurate predictions, especially on large structured datasets. |
| MLP | Multilayer Perceptron (MLP) is a neural network that captures complex nonlinear patterns between inputs and outputs and creates predictions. |
| Random Forest | Random Forest is a decision tree algorithm that improves accuracy and reduces overfitting for classification or regression tasks. |
| SVM | Support Vector Machine (SVM) is a supervised algorithm that maximizes the margin between classes to make accurate predictions, even in high-dimensional spaces. |
| XGBoost | Extreme gradient boosting (XGBoost) is an extension of GBM that's optimized for efficiency. It's a tree-based algorithm in which groups of decision trees are built sequentially to better fit the data while avoiding overfitting. |
Loss Functions for Regression Models
When you train a predictive model with a numeric outcome, loss functions measure how far predicted values are from actual values. During training, the model minimizes this loss to improve overall prediction accuracy. The selected loss function determines the types of prediction errors that the model prioritizes.
A loss function doesn't determine algorithm compatibility on its own. The combination of the selected algorithm, loss function, data, and business objective determines overall model performance. To select and configure the best algorithm for your business goal, evaluate how each loss function behaves.
Predictive AI models in Data 360 support four loss functions for regression models. The loss function that you can apply to a model depends on the selected algorithm.
| Loss Function | Description |
|---|---|
| Squared Error | Use squared error when you want larger errors to have a greater influence on the model. This loss function measures the difference between predicted and actual values by squaring the error. |
| Linear | Use linear when you want to reduce the influence of extreme errors. This loss function applies a proportional impact to errors during training, making the model less sensitive to large errors than Squared Error. |
| Quantile | Use quantile when you want to predict a specific percentile of an outcome instead of only the average. This loss function requires a target quantile. Higher quantiles emphasize avoiding underpredictions, while lower quantiles emphasize avoiding overpredictions. It determines how the model penalizes underpredictions compared with overpredictions so that it learns to predict a specific percentile, or quantile, of the outcome. |
| Gamma | Use Gamma when your target field contains many small values and only a few very large values. The target field must contain positive values and can't contain zero or negative values. This loss function predicts positive, continuous outcomes that are heavily right-skewed. |
Regression Loss Function Examples
These examples show how you can use different regression loss functions with predictive algorithms.
| Algorithm | Loss Function | Use Case |
|---|---|---|
| CatBoost, LightGBM, XGBoost, GLM, Random Forest, AdaBoost, Decision Tree, and MLP | Squared Error | Predict customer lifetime value. A prediction that's $10,000 off has a greater impact on model training than a prediction that's $1,000 off, which encourages the model to minimize larger errors. |
| CatBoost, LightGBM, XGBoost, AdaBoost, and SVM | Linear | Predict delivery times. Most deliveries are close to the expected time, but a few unusual delays can create extreme errors. Linear loss reduces the influence of those extreme errors compared with squared error. |
| CatBoost, LightGBM, and XGBoost | Quantile | Predict the 90th percentile of customer spending. Instead of estimating average spending, the model predicts a value at or below which approximately 90% of customer spending is expected to fall. |
| CatBoost, LightGBM, and XGBoost | Gamma | Predict insurance claim costs. Most claims are relatively small, while only a few are highly expensive. Gamma loss is designed for positive, right-skewed outcomes such as claim costs. |
