From 37d3f01ad97dc32e036f3dbef0b8767311e473a0 Mon Sep 17 00:00:00 2001
From: wehub-resource-sync
@@ -15,15 +19,15 @@
[](https://pypi.org/pypi/shap/)
-**SHAP (SHapley Additive exPlanations)** is a game theoretic approach to explain the output of any machine learning model. It connects optimal credit allocation with local explanations using the classic Shapley values from game theory and their related extensions (see [papers](#citations) for details and citations).
+**SHAP (SHapley Additive exPlanations)** 是一种基于博弈论的方法,用于解释任意机器学习模型的输出。它将最优信用分配与局部解释联系起来,采用博弈论中经典的 Shapley 值及其相关扩展(详见 [论文](#citations) 及引用)。
-## Install
+## 安装
-SHAP can be installed from either [PyPI](https://pypi.org/project/shap) or [conda-forge](https://anaconda.org/conda-forge/shap):
+可从 [PyPI](https://pypi.org/project/shap) 或 [conda-forge](https://anaconda.org/conda-forge/shap): 安装 SHAP:
pip install shap
@@ -31,27 +35,27 @@ pip install shap
conda install -c conda-forge shap
-### GPU support
+### GPU 支持
-To enable GPU-accelerated Tree SHAP, install from source with the CUDA toolkit available and the `SHAP_ENABLE_CUDA` environment variable set:
+要启用 GPU 加速的 Tree SHAP,请在已安装 CUDA 工具包且设置 `SHAP_ENABLE_CUDA` 环境变量的情况下从源码安装:
SHAP_ENABLE_CUDA=1 pip install .
-This requires the [CUDA toolkit](https://developer.nvidia.com/cuda-toolkit) to be installed on your system.
+这要求你的系统已安装 [CUDA toolkit](https://developer.nvidia.com/cuda-toolkit)。
-## Supported versions
+## 支持的版本
-SHAP follows [SPEC 0](https://scientific-python.org/specs/spec-0000/) for minimum supported dependency versions. We test against the versions specified there and may not fix bugs for older versions.
+SHAP 遵循 [SPEC 0](https://scientific-python.org/specs/spec-0000/) 规定的最低依赖版本。我们针对其中指定的版本进行测试,可能不会修复旧版本的 bug。
-## Contributing
+## 贡献
-We welcome contributions highly. Feel free to file an issue. Before opening a PR make sure you've read our [CONTRIBUTING.md](CONTRIBUTING.md) guideline.
+我们非常欢迎贡献。欢迎随时提交 issue。在发起 PR 之前,请确保已阅读我们的 [CONTRIBUTING.md](CONTRIBUTING.md) 指南。
-## Tree ensemble example (XGBoost/LightGBM/CatBoost/scikit-learn/pyspark models)
+## 树集成示例(XGBoost/LightGBM/CatBoost/scikit-learn/pyspark 模型)
-While SHAP can explain the output of any machine learning model, we have developed a high-speed exact algorithm for tree ensemble methods (see our [Nature MI paper](https://rdcu.be/b0z70)). Fast C++ implementations are supported for *XGBoost*, *LightGBM*, *CatBoost*, *scikit-learn* and *pyspark* tree models:
+虽然 SHAP 可以解释任意机器学习模型的输出,但我们为树集成方法开发了高速精确算法(参见我们的 [Nature MI 论文](https://rdcu.be/b0z70)). 已为 *XGBoost*、*LightGBM*、*CatBoost*、*scikit-learn* 和 *pyspark* 树模型提供快速 C++ 实现:
```python
import xgboost
@@ -74,7 +78,7 @@ shap.plots.waterfall(shap_values[0])
-If we take many force plot explanations such as the one shown above, rotate them 90 degrees, and then stack them horizontally, we can see explanations for an entire dataset (in the notebook this plot is interactive):
+若将许多如上所示的 force plot 解释旋转 90 度后水平堆叠,即可查看整个数据集的解释(在 notebook 中该图可交互):
```python
# visualize all the training set predictions
@@ -96,7 +100,7 @@ shap.plots.force(shap_values[:500])
-To understand how a single feature effects the output of the model we can plot the SHAP value of that feature vs. the value of the feature for all the examples in a dataset. Since SHAP values represent a feature's responsibility for a change in the model output, the plot below represents the change in predicted house price as the latitude changes. Vertical dispersion at a single value of latitude represents interaction effects with other features. To help reveal these interactions we can color by another feature. If we pass the whole explanation tensor to the `color` argument the scatter plot will pick the best feature to color by. In this case it picks longitude.
+要了解单个特征如何影响模型输出,我们可以绘制该特征的 SHAP 值与数据集中所有样本特征值的对比图。由于 SHAP 值表示某特征对模型输出变化的“责任”,下图展示了预测房价随纬度变化的情况。同一纬度值处的纵向分散表示与其他特征的交互效应。为揭示这些交互,可按另一特征着色。若将整个解释张量传给 `color` 参数,散点图会自动选择最佳着色特征;此处它选择了经度。
```python
# create a dependence scatter plot to show the effect of a single feature across the whole dataset
@@ -108,7 +112,7 @@ shap.plots.scatter(shap_values[:, "Latitude"], color=shap_values)
-To get an overview of which features are most important for a model we can plot the SHAP values of every feature for every sample. The plot below sorts features by the sum of SHAP value magnitudes over all samples, and uses SHAP values to show the distribution of the impacts each feature has on the model output. The color represents the feature value (red high, blue low). This reveals for example that higher median incomes increases the predicted home price.
+要概览哪些特征对模型最重要,我们可以绘制每个样本每个特征的 SHAP 值。下图按所有样本上 SHAP 值幅度之和对特征排序,并用 SHAP 值展示各特征对模型输出影响的分布。颜色表示特征值(红色为高,蓝色为低)。例如,这揭示了较高的收入中位数会提高预测房价。
```python
# summarize the effects of all the features
@@ -119,7 +123,7 @@ shap.plots.beeswarm(shap_values)
-We can also just take the mean absolute value of the SHAP values for each feature to get a standard bar plot (produces stacked bars for multi-class outputs):
+我们也可以对每个特征 SHAP 值的绝对值取平均,得到标准条形图(多类输出会生成堆叠条形图):
```python
shap.plots.bar(shap_values)
@@ -129,9 +133,9 @@ shap.plots.bar(shap_values)
-## Natural language example (transformers)
+## 自然语言示例(transformers)
-SHAP has specific support for natural language models like those in the Hugging Face transformers library. By adding coalitional rules to traditional Shapley values we can form games that explain large modern NLP model using very few function evaluations. Using this functionality is as simple as passing a supported transformers pipeline to SHAP:
+SHAP 对 Hugging Face transformers 库中的自然语言模型等提供专门支持。通过在传统 Shapley 值上添加联盟规则(coalitional rules),我们可以构造仅需极少函数评估即可解释大型现代 NLP 模型的博弈。使用该功能非常简单,只需将受支持的 transformers pipeline 传给 SHAP:
```python
import transformers
@@ -152,9 +156,9 @@ shap.plots.text(shap_values[0, :, "POSITIVE"])
-## Deep learning example with DeepExplainer (TensorFlow/Keras models)
+## 使用 DeepExplainer 的深度学习示例(TensorFlow/Keras 模型)
-Deep SHAP is a high-speed approximation algorithm for SHAP values in deep learning models that builds on a connection with [DeepLIFT](https://arxiv.org/abs/1704.02685) described in the SHAP NIPS paper. The implementation here differs from the original DeepLIFT by using a distribution of background samples instead of a single reference value, and using Shapley equations to linearize components such as max, softmax, products, divisions, etc. Note that some of these enhancements have also been since integrated into DeepLIFT. TensorFlow models and Keras models using the TensorFlow backend are supported (there is also preliminary support for PyTorch):
+Deep SHAP 是深度学习模型中 SHAP 值的高速近似算法,建立在 SHAP NIPS 论文中所述与 [DeepLIFT](https://arxiv.org/abs/1704.02685) 的联系之上。此处的实现与原始 DeepLIFT 的不同之处在于:使用背景样本分布而非单一参考值,并使用 Shapley 方程对 max、softmax、乘积、除法等组件进行线性化。请注意,其中一些增强此后也已集成到 DeepLIFT 中。支持 TensorFlow 模型以及使用 TensorFlow 后端的 Keras 模型(对 PyTorch 也有初步支持):
```python
# ...include code from https://github.com/keras-team/keras/blob/master/examples/demo_mnist_convnet.py
@@ -179,12 +183,12 @@ shap.image_plot(shap_values, -x_test[1:5])
-The plot above explains ten outputs (digits 0-9) for four different images. Red pixels increase the model's output while blue pixels decrease the output. The input images are shown on the left, and as nearly transparent grayscale backings behind each of the explanations. The sum of the SHAP values equals the difference between the expected model output (averaged over the background dataset) and the current model output. Note that for the 'zero' image the blank middle is important, while for the 'four' image the lack of a connection on top makes it a four instead of a nine.
+上图解释了四张不同图像对应的十个输出(数字 0-9)。红色像素会提高模型的输出,蓝色像素会降低输出。输入图像显示在左侧,并作为各张解释图背后近乎透明的灰度背景。SHAP 值的总和等于预期模型输出(在背景数据集上取平均)与当前模型输出之间的差值。请注意,对于“零”的图像,中间空白区域很重要;而对于“四”的图像,顶部缺少连接才使它被识别为四而不是九。
-## Deep learning example with GradientExplainer (TensorFlow/Keras/PyTorch models)
+## 使用 GradientExplainer 的深度学习示例(TensorFlow/Keras/PyTorch 模型)
-Expected gradients combines ideas from [Integrated Gradients](https://arxiv.org/abs/1703.01365), SHAP, and [SmoothGrad](https://arxiv.org/abs/1706.03825) into a single expected value equation. This allows an entire dataset to be used as the background distribution (as opposed to a single reference value) and allows local smoothing. If we approximate the model with a linear function between each background data sample and the current input to be explained, and we assume the input features are independent then expected gradients will compute approximate SHAP values. In the example below we have explained how the 7th intermediate layer of the VGG16 ImageNet model impacts the output probabilities.
+Expected gradients(预期梯度)将 [Integrated Gradients](https://arxiv.org/abs/1703.01365), SHAP,以及 [SmoothGrad](https://arxiv.org/abs/1706.03825) 融合到一个单一的期望值方程中。这使得可以将整个数据集用作背景分布(而非单个参考值),并支持局部平滑。若我们用每个背景数据样本与当前待解释输入之间的线性函数来近似模型,并假设输入特征相互独立,则 expected gradients 将计算出近似的 SHAP 值。在下面的示例中,我们解释了 VGG16 ImageNet 模型的第 7 个中间层如何影响输出概率。
```python
from keras.applications.vgg16 import VGG16
@@ -227,11 +231,11 @@ shap.image_plot(shap_values, to_explain, index_names)
-Predictions for two input images are explained in the plot above. Red pixels represent positive SHAP values that increase the probability of the class, while blue pixels represent negative SHAP values the reduce the probability of the class. By using `ranked_outputs=2` we explain only the two most likely classes for each input (this spares us from explaining all 1,000 classes).
+上图解释了两张输入图像的预测结果。红色像素表示正值 SHAP,会提高该类别的概率;蓝色像素表示负值 SHAP,会降低该类别的概率。通过使用 `ranked_outputs=2`,我们仅解释每个输入中最可能的两个类别(这样就不必解释全部 1,000 个类别)。
-## Model agnostic example with KernelExplainer (explains any function)
+## 使用 KernelExplainer 的模型无关示例(可解释任意函数)
-Kernel SHAP uses a specially-weighted local linear regression to estimate SHAP values for any model. Below is a simple example for explaining a multi-class SVM on the classic iris dataset.
+Kernel SHAP 使用特殊加权的局部线性回归来估计任意模型的 SHAP 值。下面是一个简单示例,用于解释经典 iris 数据集上的多分类 SVM。
```python
import sklearn
@@ -257,9 +261,9 @@ shap.force_plot(explainer.expected_value[0], shap_values[0][0,:], X_test.iloc[0,
-The above explanation shows four features each contributing to push the model output from the base value (the average model output over the training dataset we passed) towards zero. If there were any features pushing the class label higher they would be shown in red.
+上述解释展示了四个特征各自如何将模型输出从基准值(我们传入的训练数据集上的平均模型输出)推向零。若有任何特征将类别标签推高,则会以红色显示。
-If we take many explanations such as the one shown above, rotate them 90 degrees, and then stack them horizontally, we can see explanations for an entire dataset. This is exactly what we do below for all the examples in the iris test set:
+若将许多如上所示的解释旋转 90 度,再水平堆叠,便可以看到整个数据集的解释。我们在下面正是这样做的,针对 iris 测试集中的所有样本:
```python
# plot the SHAP values for the Setosa output of all instances
@@ -269,67 +273,67 @@ shap.force_plot(explainer.expected_value[0], shap_values[0], X_test, link="logit
-## SHAP Interaction Values
+## SHAP Interaction Values(SHAP 交互值)
-SHAP interaction values are a generalization of SHAP values to higher order interactions. Fast exact computation of pairwise interactions are implemented for tree models with `shap.TreeExplainer(model).shap_interaction_values(X)`. This returns a matrix for every prediction, where the main effects are on the diagonal and the interaction effects are off-diagonal. These values often reveal interesting hidden relationships, such as how the increased risk of death peaks for men at age 60 (see the NHANES notebook for details):
+SHAP 交互值是 SHAP 值向高阶交互的推广。对于树模型,已使用 `shap.TreeExplainer(model).shap_interaction_values(X)` 实现了成对交互的快速精确计算。这会为每次预测返回一个矩阵,其中主效应位于对角线,交互效应位于非对角线。这些值常常揭示有趣的隐含关系,例如男性死亡风险在 60 岁时达到峰值(详见 NHANES notebook):