Ir al contenido

SHAP Fundamentals (The 'What' and the 'Why')

Introduction to the theoretical concepts of SHAP (SHapley Additive exPlanations) and black-box model interpretability in Portelcore.

0views

In the analytical ecosystem of Portelcore, advanced predictive models such as Gradient Boosting (XGBoost, LightGBM) or Deep Neural Networks frequently act as “black box” models (Black Box). Although these algorithms achieve exceptional precision, their inherent complexity prevents directly deciphering the underlying logic behind each prediction.

SHAP (SHapley Additive exPlanations) is a unified interpretability framework that allows for auditing and deciphering these predictive decisions in a fair, mathematical, and transparent manner.

Based on cooperative game theory and the contribution of Shapley Values (whose creator, Lloyd Shapley, received the Nobel Prize in Economics), SHAP poses a fundamental question applicable to Machine Learning:

If a model generates a specific prediction, what is the “fair contribution” of each predictive variable (feature) in deviating that result from the average prediction?

graph TD
subgraph Explicación Local de SHAP
Base["Valor Base (Promedio Global)<br>E[f(x)]"] -->|"+15% (Retención de Sesiones)"| C1["Contribución Positiva 1"]
Base -->|"+20% (Historial de Conversión)"| C2["Contribución Positiva 2"]
Base -->|"-10% (Latencia de Red)"| C3["Contribución Negativa 1"]
C1 --> Sum["Suma de Atribuciones SHAP<br>(+25%)"]
C2 --> Sum
C3 --> Sum
Sum --> Final["Predicción Final<br>f(x)"]
end
style Base fill:#2e3440,stroke:#81a1c1,stroke-width:2px,color:#d8dee9
style Final fill:#a3be8c,stroke:#8fbcbb,stroke-width:2px,color:#2e3440
style Sum fill:#3b4252,stroke:#4c566a,stroke-width:1px,color:#d8dee9

The primary objective when integrating SHAP into our data workflows is to provide predictions with confidence and traceability. Ensuring that model logic is aligned with business rules is a critical engineering priority.

Our analytical processes implement SHAP based on four fundamental pillars:

  • 1. Trust and Regulatory Compliance:
    • Verify that predictive models are not biased by spurious variables or circumstantial “noise” from the data environment.
    • Provide clear technical justifications to governance and audit committees regarding the origin of an automated decision.
  • 2. Audit and Debugging:
    • It acts as the premier diagnostic tool for predictive errors.
    • If the model issues an erroneous or atypical prediction, SHAP accurately breaks down which variables caused the estimator's behavior to deviate.
  • 3. Clinical/Commercial Insights Discovery:
    • Identifies complex correlations and non-linear behaviors that traditional statistical methods often overlook.
    • Allows for understanding cross-interactions (for example, how the effect of the variable Age is modulated by Usage Frequency).
  • 4. Directive Feature Engineering:
    • By measuring the direct and quantifiable impact of each variable on real performance, the development team can discard irrelevant variables, optimizing model speed and reducing the dimensionality of the dataset.

A SHAP explanation does not produce simple qualitative binary classifications, but rather delivers an exact additive breakdown. For any individual sample $x$, the prediction is composed of:

1. The Base Value (Base Value - $E[f(x)]$)

Sección titulada «1. El Valor Base (Base Value - $E[f(x)]$)»

It is the statistical average of the model predictions over the training set. It is the default response that the estimator would emit if it did not have information or features from the record to be predicted.

Example: “The historical average probability of a customer leak in Portelcore is 14.5%”

It is the real impact measured on the model's output scale for each feature.

  • Positive Impact ($\phi_i > 0$): Pushes the prediction upward, above the base mean (visually represented in warm tones).
  • Negative Impact ($\phi_i < 0$): Reduces the prediction, dragging it down below the base mean (visually represented in cool tones).

The final predicted value for the specific sample is the mathematical sum of its component parts:

$$\text{Final Prediction } f(x) = \text{Base Value } E[f(x)] + \sum_{i=1}^{M} \phi_i$$

Where $M$ is the total number of features in the model.


Before moving on to code integration, verify that you master the following key interpretability concepts:

  • I understand that the Base Value represents the average model prediction in the absence of sample-specific data. ✓
  • I understand the fundamental additive property: the sum of all SHAP values and the base value is mathematically equal to the final prediction. ✓
  • I correctly differentiate between local explanations (oriented toward understanding a single record) and global explanations (oriented toward auditing the entire model). ✓