SIGMA: Self-Guided Integrated Gradient Method for Attribution
Modern neural networks can have very high confidence — but confidence alone does not tell us why a prediction was made.
Feature attribution methods aim to answer this why by highlighting which parts of an input most influenced a model’s decision. In practice, however, the way we choose to explain a prediction can actually shape the explanation itself.
What does it mean to explain a prediction?
Given an input and a trained model, an attribution method assigns importance scores to individual input features — pixels in an image, time points on a graph, or tokens in text — indicating how strongly each contributed to the final prediction.
Among the most widely used approaches are path-based methods, which accumulate model gradients as the input is gradually transformed, telling us how sensitive each input feature is to the output prediction.
The baseline assumption
Integrated Gradients explains a prediction by integrating gradients along a straight path from a baseline input to the image being explained. The baseline is intended to represent the absence of information — often a black or blurred image. However, different baselines can lead to noticeably different explanations, even when the model and input remain unchanged.
Saturation along attribution paths
As the input moves away from the baseline, the model’s confidence often increases rapidly before entering a saturated region where further changes have little effect on the output. Gradients accumulated in these flat regions can dominate the final attribution, despite contributing little to the actual decision.
Thinking in terms of confidence landscapes
Rather than viewing attribution as interpolation between two images, it can be helpful to think of the model’s output as a confidence landscape over input space. An informative explanation should focus on the regions where the model’s belief actually changes — not where it is already certain.
A self-guided path
SIGMA removes the need for a baseline entirely. Instead of following a predefined path, SIGMA constructs its own trajectory by iteratively perturbing the input in directions that reduce the model’s confidence in the predicted class. Gradients are accumulated along this path and weighted by the corresponding drop in confidence.
What changes in practice?
Because SIGMA follows the model’s confidence downhill, it avoids early saturation and continues to collect informative gradients throughout the path. The resulting attribution maps tend to be more spatially coherent and less influenced by regions that have little effect on the prediction.
Faithfulness to model behaviour
By accumulating gradients in proportion to confidence change, SIGMA aligns attribution strength with the model’s sensitivity. Revealing input features in order of attribution importance reconstructs the model’s confidence more efficiently than random or saturated-gradient explanations.
Beyond explanation
Following confidence collapse naturally produces perturbed inputs that remain recognisable to humans but receive near-zero confidence from the model. These low-confidence variants expose decision boundaries and can be reused as training augmentations, linking interpretability and model reliability.
Conclusion
SIGMA offers a shift in perspective for attribution based interpretability — from explaining predictions relative to arbitrary references, to explaining them by following the model’s own confidence. By treating explanations as paths shaped by the model itself, we gain both clearer attributions and a deeper understanding of how confidence emerges and collapses.
Paper: Preprint on TechRxiv
Code: GitHub repository
Contact: sjh9@hw.ac.uk
