TL;DR

A tied event time occurs when two or more individuals have the same recorded event time, for example two patients both developing heart failure 10 days after discharge.

The main practical lesson is:

Tie handling depends on the training objective, not just the model name.

For a Cox partial-likelihood model, ties create an ambiguity in the risk set and require a rule such as Breslow, Efron, or exact partial likelihood. For pairwise ranking losses, equal event times usually do not provide an ordering constraint. For parametric or discrete-time likelihoods, repeated event times are often ordinary observations rather than a special problem.

In clinical datasets, ties are common when follow-up is recorded in integer days, dates are rounded, or cohorts are large. The safest workflow is therefore to identify the objective being optimized and then verify how the software actually handles equal times.

1. What Is a Tie?

Suppose two prediction episodes have

meaning both experience the endpoint on day 10.

This is an event-time tie. It is different from:

• two patients receiving the same predicted risk score;
• competing events such as heart failure versus death;
• repeated episodes from the same patient;
• censoring, which determines whether the event time is observed at all.

These distinctions matter because each creates a different statistical problem.

A further edge case is an event–censor tie, where one patient has an event and another is censored at the same recorded time. In the usual Cox risk-set convention, a patient censored at time remains in for failures at ; this effectively places events before censoring at the same recorded time. Evaluation metrics may use different comparability conventions, so event–censor ties should be checked separately rather than treated as ordinary event–event ties.

A useful question is not simply “Does my dataset contain ties?” but rather:

Does my loss function require a strict ordering between these observations?

That question determines whether any special treatment is needed.

2. Why Ties Are Common in Clinical Data

In theory, survival time can be continuous. In practice, clinical data are often stored at much lower resolution:

• follow-up may be measured in whole days;
• event timestamps may be rounded to dates;
• prediction time may be defined from discharge;
• administrative pipelines may truncate or discretize timestamps;
• large cohorts naturally produce more repeated times.

Thus, two events recorded on day 10 do not necessarily mean that they occurred at exactly the same instant. They may simply be indistinguishable at the available time resolution.

This becomes important for models whose objective depends on the order in which patients leave a risk set.

3. Why Cox Models Need a Tie Rule

For a Cox proportional hazards model, let

be the log-risk score. With a unique event time , the partial likelihood compares the patient who experiences the event with everyone still at risk:

Now suppose patients and both experience the event on day 10.

The first event should be evaluated using the full risk set. For the second event, however, the first patient should already have left that set. The problem is that the recorded data do not tell us whether or occurred first.

That is the Cox tie problem.

Breslow, Efron, and Exact

Let be the set of events observed at time , and define

The three standard approaches in survival::coxph() are:

For two tied events, Breslow uses denominators

whereas Efron uses

For tied events, Efron contributes

The R survival documentation uses Efron as the default for ordinary single-state Cox models and notes that exact partial likelihood can become computationally expensive when many subjects are tied at one time point. A multistate Cox model is an important exception: in survival 3.8-11, the default is Breslow because a defensible general extension of Efron is more complicated, and the current Efron implementation is effectively limited to tied transitions of the same type.

Importantly, “exact” does not mean better prediction. It means a more exact treatment of the partial likelihood under a particular discrete-time interpretation.

4. DeepSurv Is Still a Cox Model

DeepSurv replaces the linear Cox predictor

with a neural network

but the objective is still based on the Cox partial likelihood.

Therefore, the tie question does not disappear just because the model is deep.

This leads to an important implementation rule:

Never infer the tie method from the label “Cox loss” or “DeepSurv.” Check the actual implementation.

For example, TorchSurv 0.2.0 exposes ties_method="efron" or "breslow" for its Cox loss, with Efron as the default. In contrast, the pycox 0.3.0 CoxPH loss sorts observations by duration and computes the denominator using a cumulative sum rather than explicitly constructing tied risk sets. Its source does not show a documented Efron or Breslow correction in that loss. From this implementation, equal-duration observations can be inferred to depend on their within-tie ordering; this is an interpretation of the version-pinned source rather than an explicit library claim.

So two neural Cox implementations can optimize slightly different objectives even when both are described as “DeepSurv.”

5. Pairwise Ranking Losses Have a Different Tie Problem

A pure ranking objective asks whether one patient should be assigned higher risk than another.

For example, a comparable-pair set may be defined as

If

then the observation supports the ordering

But if

the observed data do not tell us whether should rank above or vice versa.

The clean statistical response is therefore usually not to create a strict pairwise constraint between them.

This is conceptually different from Efron. Efron approximates an unknown ordering inside a Cox risk-set likelihood. A pairwise ranking loss can simply say that an equal-time pair carries no information about which patient should rank first.

DeepHit

DeepHit is a useful example because its objective combines two components:

The original paper defines acceptable ranking pairs using a strict time inequality . Equal observed times therefore do not define an ordered acceptable pair for the ranking term.

At the same time, DeepHit models a discrete event-time distribution, so multiple patients occupying the same time bin are perfectly valid observations for its likelihood component.

One model can therefore treat ties differently in different parts of its objective.

Survival SVM

Survival SVMs provide another ranking example. Conceptually, equal event times do not encode a strict temporal ordering. However, FastSurvivalSVM in scikit-survival 0.28.0 resolves ties in survival times before optimization, using random_state to make this ordering reproducible. This implementation convention should not be mistaken for temporal information observed in the data.

6. Some Models Do Not Need Cox-Style Tie Correction

The simplest way to see this is to compare the training objectives.

Weibull and other parametric models

For a Weibull model, an uncensored patient contributes a density term and a censored patient contributes a survival term:

If two patients both experience an event at time 10, the likelihood simply contains

There is no unknown risk-set departure order to approximate.

Discrete-time models

For models such as LogisticHazard, PMF, or MTLR, the time axis is represented by intervals or discrete bins. Multiple patients experiencing an event in the same interval are therefore part of the model definition rather than an exceptional case.

This distinction is useful:

Repeated event times are observations. They become a special “tie problem” only when the objective needs information that the recorded time resolution does not provide.

7. A Small Clinical Example

Suppose five patients are still at risk immediately before day 10:

Patients and both experience heart failure on day 10.

Cox model

The model must evaluate two event contributions while knowing only that both occurred at the same recorded time.

• Breslow keeps the same denominator for both.
• Efron progressively adjusts the denominator.
• Exact partial likelihood sums over the possible tied-event subsets consistent with the observed tie.

Pairwise ranking model

The data support statements such as

and therefore an ordering between and .

They do not support a strict ordering between and , because

Weibull or discrete-time model

Both patients can contribute normally at time 10 or in the same time bin. No Cox-specific tie approximation is required.

The same raw data therefore create three different behaviors because the objectives ask three different questions.

8. Training Ties Are Not Evaluation Ties

There is one final source of confusion: the concordance index also contains tie rules.

For Harrell-style concordance, two patients who both experience an event at the same observed time generally do not form an ordered comparable event-event pair. For example, scikit-survival 0.28.0 treats equal event times as non-comparable and assigns 0.5 credit when a comparable pair has tied predicted risk scores.

So a paper may need to report two distinct decisions:

1. how tied event times were handled during model fitting; and
2. how tied observed times and tied predicted risks were handled during evaluation.

These should not be conflated.

9. Practical Decision Guide

When you encounter many tied event times, ask:

1. What objective am I optimizing?
   Cox partial likelihood, pairwise ranking, full parametric likelihood, discrete likelihood, or something else?

2. Does that objective require a strict event ordering?
   If not, a Cox-style tie correction may be irrelevant.

3. Is time truly discrete, or merely recorded coarsely?
   This affects whether an exact discrete interpretation is scientifically meaningful.

4. How frequent are the ties?
   For Cox models, the difference between approximations matters more as tie multiplicity increases.

5. What does the software actually implement?
   Do not assume that every package using the phrase “Cox loss” is using Efron.

6. Does the evaluation metric have its own tie convention?
   Training-time and evaluation-time ties are separate issues.

For a standard single-event Cox model with ordinary clinical follow-up data, Efron is a strong default. For other survival objectives, first determine whether there is actually a tie problem to solve.

10. Reporting Recommendation

For a Cox model, a concise Methods sentence is:

Tied event times were handled using the Efron approximation to the Cox partial likelihood.

For a neural Cox model, it is better to add the implementation:

The neural risk model was trained using a Cox partial-likelihood objective with Efron handling of tied event times.

For a broader reproducibility statement, report the software package, version, objective, and tie method explicitly.

Takeaways

The key distinction is not classical versus deep survival analysis. It is what information the loss function requires from the observed event times.

• Cox models need a rule because tied events make risk-set departure order ambiguous.
• Pairwise ranking objectives should not invent an ordering that the data do not contain.
• Parametric and discrete-time likelihoods can usually accept repeated event times directly.
• Software behavior is part of the statistical method and should be verified and reported.
• Tie handling during training is separate from tie handling in concordance-based evaluation.

Once this distinction is clear, tied event times become much less mysterious.

References

Original and classical methods

1. Cox DR. Regression Models and Life-Tables. Journal of the Royal Statistical Society: Series B. 1972;34(2):187–220.
   https://doi.org/10.1111/j.2517-6161.1972.tb00899.x

2. Breslow NE. Covariance Analysis of Censored Survival Data. Biometrics. 1974;30(1):89–99.
   https://doi.org/10.2307/2529620

3. Efron B. The Efficiency of Cox’s Likelihood Function for Censored Data. Journal of the American Statistical Association. 1977;72(359):557–565.
   https://doi.org/10.1080/01621459.1977.10480613

4. Therneau TM, Grambsch PM. Modeling Survival Data: Extending the Cox Model. Springer; 2000.

5. Katzman JL, Shaham U, Cloninger A, Bates J, Jiang T, Kluger Y. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Medical Research Methodology. 2018;18:24.
   https://doi.org/10.1186/s12874-018-0482-1

6. Lee C, Zame WR, Yoon J, van der Schaar M. DeepHit: A Deep Learning Approach to Survival Analysis With Competing Risks. AAAI. 2018;32(1).
   https://doi.org/10.1609/aaai.v32i1.11842

7. Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS. Random Survival Forests. The Annals of Applied Statistics. 2008;2(3):841–860.
   https://doi.org/10.1214/08-AOAS169

Software behavior checked for this note

8. R survival 3.8-11, coxph() documentation. Efron is the default for ordinary single-state Cox models; multistate models default to Breslow.
   https://stat.ethz.ch/R-manual/R-devel/library/survival/html/coxph.html

9. TorchSurv 0.2.0, Cox loss documentation. Supports ties_method="efron" and "breslow"; Efron is the default.
   https://opensource.nibr.com/torchsurv/_autosummary/torchsurv.loss.cox.html

10. pycox 0.3.0, CoxPH loss source. The loss sorts by duration and uses a cumulative-sum denominator rather than explicitly constructing tied risk sets.
    https://github.com/havakv/pycox/blob/v0.3.0/pycox/models/loss.py

11. scikit-survival 0.28.0, FastSurvivalSVM and concordance_index_censored documentation.
    https://scikit-survival.readthedocs.io/en/v0.28.0/api/generated/sksurv.svm.FastSurvivalSVM.html
    https://scikit-survival.readthedocs.io/en/v0.28.0/api/generated/sksurv.metrics.concordance_index_censored.html

Software behavior and versions were checked on September 7, 2026.

我的知乎版本, 貌似知乎的latex公式反馈更好

Brier Score (mean squared error) 感觉这个 跟L2 loss 很像。但是因为我最近在做比较多survival analysis，所以经常接触 brier score。所以看到很少有人整理这块，我先简单整理，以后会多次更新修改。
1 - Brier Score
Brier Score 最原始的公式:

$ BS = \frac{1}{N} \sum_{t=1}^N(\hat y_t - y_t)^2 $,

L2 loss 对比

$ \text{L2 loss} = {\sum_{t=1}^N (y_t - \hat y_t)^2} $

N: 是总共检测的样本数目
y_hat: 是 probability of y, 也就是 预测的概率
y: 是 ground truth 也就是 真实的y
Brier Score 的出来的范围 是 [0,1] 之间，然后Brier Score 越小则模型准确率越高。
Code:

1
2
3
4
5
6

import numpy as np
from sklearn.metrics import brier_score_loss
y_true = np.array([0, 0, 1, 1, 1])
y_prob = np.array([0.2, 0.1, 0.8, 0.9, 0.5])

brier_score_loss(y_true, y_prob) # 0.07

2 - Brier Score 在生存测试 (without censor) [1]
在生存测试中，我们预测的是每个独立的病人的生存概率。

添加图片注释，不超过 140 字（可选）
也就是在时间点T = t’，病人A的生存概率 S(t’, X_A).
那么在每个时间点T 的生存概率就可以与病人的真实生存情况得出 Brier Score 在时间点T = t.

$ \mathbb{1} = T_i > t $

$ BS(t) = \frac{1}{N} \sum_{i=1}^{N} (\mathbb{1} - \hat S (t, \vec x_i))^2 $

$\mathbb{1}$：indicator function 也就是 $T_i > t$ 就是1 不然就是0

$\vec x_i$ ：因为每个病人其实是feature vector

bench_mark: 当所有的 S(x) = 0.5, BS = 1/N (N x 0.5^2 ) = 0.25
3 - Brier Score 在生存测试 (with censor)
censor 就是删失病人。在某个时间点 Ti 之后，病人数据消失了，我们只知道病人在Ti 之前是活着的。
这个时候我们引入了一个新的概念IPCW
$\hat G(t) = \prod_j \frac{n_j - d^{\star}_j}{n_j}: j \in {V_j < t} $ 用来转移censor 病人的weight 到 uncensor 到病人去评估准确度。

$n_j$: 所有 在 t 时有censor的可能性的 病人

$d^{\star}_j$: 在 t 时censor 的病人
例子

这里有个很有意思的发现，组里一个phd 大佬点醒了我们
我们会发现在算S3 的 Brier Score 的时候 weight 是 4/3，是因为除了他本身的 1，还加了 S2的 1/3。而算S5的 Brier Score 的时候 weight 是 8/3，除了S5 的1 以外 还加了 S4 的1 以及 S3的 2/3，也就是说 S2 的weight 有1/3 给了 S3， 有2/3 给了S5.
当时疑惑了好久，然后后来他发现 并不是 S2的weight 1/3 给了S5 而是S2的weight uniform 给了后面所有的 S (S3, S4, S5 各有1/3)。然而 因为 S4 本身也是 删失的情况，所以S4 连带着 S2的1/3 一起给了S5 所以才导致 S5的weight 是8/3。
然后我们提出了几个疑问，
1：万一最后全是censor 删失病人怎么办，后面的删失病人 的weight 没办法transfer 到最后一个 uncensor 病人上。(后来我们查论文得知 最后一个病人一定是uncensor 病人)
2：会不会导致越往后的病人 weight 越重要，当censor 病人足够多的时候，最后一个病人的预测情况可以左右结果。(这个好像的确是一个问题 但不知道这个是好是坏)

在融入 $ \hat G(t) $ 之后, BS 可以更新为

$ BS(t) = \frac{1}{N} \sum_{i = 1}^{N} \left( \frac{\left( 0 - \hat{S}(t, \vec{x}i)\right)^2 \cdot \mathbb{1}{T_i \leq t, \delta_i = 1}}{ \hat{G}(T_i^-)} + \frac{ \left( 1 - \hat{S}(t, \vec{x}i)\right)^2 \cdot \mathbb{1}{T_i > t}}{ \hat{G}(t)} \right) $

$ BS(t) = \frac{1}{N} \sum_{i = 1}^N \left{ \begin{array}{lr} \frac{0-\hat S(t|Z_i)^2}{\hat G(t_i)} & \text{if } t_i \leq t, \delta_i = 1 \ \frac{(1 - \hat S(t|Z_i))^2}{\hat G(t} & \text{if } t_i > t \ 0 & \text{ if } t_i = t, \delta_i = 0 \end{array} \right. $

4 - integral Brier Score [2]
因为 BS(t) 是只考虑在 时间点 t 的时候 BS 是多少，但是生存概率是一个 时间范围内，如果我们想知道整个时间段的生存概率是否正确，我们可以用IBS
当我们只考虑 uncensor 病人，也就是有event的病人：

$ IBS(\tau, V_U, \hat S(\cdot | \cdot)) = \frac{1}{\tau} \int_{0}^{\tau} BS_t (V_U, \hat S(t| \cdot)) dt $

如果我们要加上 censor 的病人，我们就要用我们之前的IPCW 的weight 了

$ IBS(\tau, V, \hat S(\cdot | \cdot)) = \frac{1}{\tau} \int_{0}^{\tau} \left( \frac{\left( 0 - \hat{S}(t, \vec{x}i)\right)^2 \cdot \mathbb{1}{T_i \leq t, \delta_i = 1}}{ \hat{G}(T_i^-)} + \frac{ \left( 1 - \hat{S}(t, \vec{x}i)\right)^2 \cdot \mathbb{1}{T_i > t}}{ \hat{G}(t)} \right) dt $

Reference:
[1] Graf, Erika, Claudia Schmoor, Willi Sauerbrei, and Martin Schumacher. 1999. “Assessment and Comparison of Prognostic Classification Schemes for Survival Data.”Statistics in Medicine18 (17-18): 2529–45.
[2] Haider, H., Hoehn, B., Davis, S., & Greiner, R. (2020). Effective ways to build and evaluate individual survival distributions.Journal of Machine Learning Research,21(85), 1-63.

Partial Dependence Plots

部分特征图

While feature importance shows what variables most affect predictions, partial dependence plots show how a feature affects predictions.

当特征重要性显示出哪一个特征对预测结果影响最大，部分特征图显示出一个特征如何影响预测结果。

This is useful to answer questions like:

这对回答以下问题非常有帮助。

Controlling for all other house features, what impact do longitude and latitude have on home prices? To restate this, how would similarly sized houses be priced in different areas?

在其他特征都相同的情况下，经纬度是如何影响房价? 换句话说在不同的地区，相同的房子的价格是如何不同的?

Are predicted health differences between two groups due to differences in their diets, or due to some other factor?

两个人群中，健康因素是由于饮食不同而变化，还是由于其他因素?

If you are familiar with linear or logistic regression models, partial dependence plots can be interepreted similarly to the coefficients in those models. Though, partial dependence plots on sophisticated models can capture more complex patterns than coefficients from simple models. If you aren’t familiar with linear or logistic regressions, don’t worry about this comparison.

如果你对线性回归或者逻辑回归模型熟悉的话，部分特征图可以解释为这些模型特征前面的系数。但是部分特征图可以解决更复杂的模型，而不局限于只是简单模型中的系数。如果你不熟悉线性回归或者逻辑回归模型，也不必担心。

We will show a couple examples, explain the interpretation of these plots, and then review the code to create these plots.

我们会展示几个案例，然后去解释这些图形包括生成这些图形的代码。

How it Works

如何运行

Like permutation importance, partial dependence plots are calculated after a model has been fit. The model is fit on real data that has not been artificially manipulated in any way.

和排列重要性类似，部分特征图也必须在模型训练结束之后开始计算。这个模型必须完全基于真实数据（不能有人为的更改）。

In our soccer example, teams may differ in many ways. How many passes they made, shots they took, goals they scored, etc. At first glance, it seems difficult to disentangle the effect of these features.

在我们的足球例子中，每个足球队都有很多方面不同。比如多少次传球，射门，得分等。一开始，我们会觉得很难去解开每个特征对结果的影响。

To see how partial plots separate out the effect of each feature, we start by considering a single row of data. For example, that row of data might represent a team that had the ball 50% of the time, made 100 passes, took 10 shots and scored 1 goal.

我们可以从一条数据开始去了解部分特征图如何去解决每个特征的影响的。比如这一条数据，这个队伍控球50%，传了100次，射了10次门，并且得了1分。

We will use the fitted model to predict our outcome (probability their player won “man of the game”). But we repeatedly alter the value for one variable to make a series of predictions. We could predict the outcome if the team had the ball only 40% of the time. We then predict with them having the ball 50% of the time. Then predict again for 60%. And so on. We trace out predicted outcomes (on the vertical axis) as we move from small values of ball possession to large values (on the horizontal axis).

我们会用训练好的模型去预测结果(大概是哪位队员会赢得本场比赛的mvp)。但是我们重复改变其中的一个特征来得出一系列预测。我们可以预测如果一个队只控球40%比赛时间。我们继续预测如果一个队只控球50%比赛时间，然后预测60%，以此类推。我们把预测结果放Y轴，然后我们在X轴上把控球从小往大移动。

In this description, we used only a single row of data. Interactions between features may cause the plot for a single row to be atypical. So, we repeat that mental experiment with multiple rows from the original dataset, and we plot the average predicted outcome on the vertical axis.

解释中我们只用了一条数据。得出来的结果并不是最典型的。所以我们重复这种思想实验并且更改原来的数据。我们可以画出预测结果的平均数在Y轴上。

Code Example

Model building isn’t our focus, so we won’t focus on the data exploration or model building code.

1
2
3
4
5
6
7
8
9
10
11
12

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.tree import DecisionTreeClassifier

data = pd.read_csv('../input/fifa-2018-match-statistics/FIFA 2018 Statistics.csv')
y = (data['Man of the Match'] == "Yes")  # Convert from string "Yes"/"No" to binary
feature_names = [i for i in data.columns if data[i].dtype in [np.int64]]
X = data[feature_names]
train_X, val_X, train_y, val_y = train_test_split(X, y, random_state=1)
tree_model = DecisionTreeClassifier(random_state=0, max_depth=5, min_samples_split=5).fit(train_X, train_y)

For the sake of explanation, our first example uses a Decision Tree which you can see below. In practice, you’ll use more sophistated models for real-world applications.

我们的第一个例子用了决策树。在现实中你会遇到更复杂的模型。

1
2
3
4
5

from sklearn import tree
import graphviz

tree_graph = tree.export_graphviz(tree_model, out_file=None, feature_names=feature_names)
graphviz.Source(tree_graph)

As guidance to read the tree:

• Leaves with children show their splitting criterion on the top
• The pair of values at the bottom show the count of True values and False values for the target respectively, of data points in that node of the tree.

为了更方便去理解决策树

• 叶子上面的解释了如何从顶端分离下来的标准
• 在最底部上面的值表示了正确还是错误的个数分别有多少

Here is the code to create the Partial Dependence Plot using the PDPBox library.

1
2
3
4
5
6
7
8
9

from matplotlib import pyplot as plt
from pdpbox import pdp, get_dataset, info_plots

# Create the data that we will plot
pdp_goals = pdp.pdp_isolate(model=tree_model, dataset=val_X, model_features=feature_names, feature='Goal Scored')

# plot it
pdp.pdp_plot(pdp_goals, 'Goal Scored')
plt.show()

Here is another example plot:

1
2
3
4
5
6
7
8
9

from matplotlib import pyplot as plt
from pdpbox import pdp, get_dataset, info_plots

# Create the data that we will plot
pdp_goals = pdp.pdp_isolate(model=tree_model, dataset=val_X, model_features=feature_names, feature='Goal Scored')

# plot it
pdp.pdp_plot(pdp_goals, 'Goal Scored')
plt.show()

A few items are worth pointing out as you interpret this plot

• The y axis is interpreted as change in the prediction from what it would be predicted at the baseline or leftmost value.

• A blue shaded area indicates level of confidence

有一些值得在你图上解释的点：

• Y 轴被解释成从baseline 或者最左边的值开始 预测上的变量

• 蓝色阴影代表了不同程度的置信区间

From this particular graph, we see that scoring a goal substantially increases your chances of winning “Player of The Game.” But extra goals beyond that appear to have little impact on predictions.

从这一部分的图，我们不难发现进球得分基本上会增加你得本场比赛MVP的概率。但是多出来的进球反而对增加这个概率帮助并不是太大。

Here is another example plot:

1
2
3
4
5

feature_to_plot = 'Distance Covered (Kms)'
pdp_dist = pdp.pdp_isolate(model=tree_model, dataset=val_X, model_features=feature_names, feature=feature_to_plot)

pdp.pdp_plot(pdp_dist, feature_to_plot)
plt.show()

This graph seems too simple to represent reality. But that’s because the model is so simple. You should be able to see from the decision tree above that this is representing exactly the model’s structure.

You can easily compare the structure or implications of different models. Here is the same plot with a Random Forest model.

这个图看起来吧现实解释得很简单。但是这是因为模型简单。你应该去关注这个是如何解释模型结构的而不局限在决策树。

你可以很容易得比较不同模型。这里是随机森林模型的图。

1
2
3
4
5
6
7

# Build Random Forest model
rf_model = RandomForestClassifier(random_state=0).fit(train_X, train_y)

pdp_dist = pdp.pdp_isolate(model=rf_model, dataset=val_X, model_features=feature_names, feature=feature_to_plot)

pdp.pdp_plot(pdp_dist, feature_to_plot)
plt.show()

This model thinks you are more likely to win Player of The Game if your players run a total of 100km over the course of the game. Though running much more causes lower predictions.

In general, the smooth shape of this curve seems more plausible than the step function from the Decision Tree model. Though this dataset is small enough that we would be careful in how we interpret any model.

在这个模型中，你会更倾向于整场比赛跑了超过100km的队员得MVP，其实这个是比较低准确率的预测。

普遍来说，这种平稳的图形比决策树模型更加模糊。我们得非常小心去解释一些小数据模型

2D Partial Dependence Plots

2维的部分特征图

If you are curious about interactions between features, 2D partial dependence plots are also useful. An example may clarify what this.

We will again use the Decision Tree model for this graph. It will create an extremely simple plot, but you should be able to match what you see in the plot to the tree itself.

如果你对特征之间如何互相影响有兴趣的话，2维的部分特征图会非常有帮助。这里有一个例子可以解释这个。

我们会重复使用决策树模型。它会创造一个简单的图形，虽然简单，但是足够我们去对比到决策树图形本身。

1
2
3
4
5
6

# Similar to previous PDP plot except we use pdp_interact instead of pdp_isolate and pdp_interact_plot instead of pdp_isolate_plot
features_to_plot = ['Goal Scored', 'Distance Covered (Kms)']
inter1  =  pdp.pdp_interact(model=tree_model, dataset=val_X, model_features=feature_names, features=features_to_plot)

pdp.pdp_interact_plot(pdp_interact_out=inter1, feature_names=features_to_plot, plot_type='contour')
plt.show()

This graph shows predictions for any combination of Goals Scored and Distance covered.

For example, we see the highest predictions when a team scores at least 1 goal and they run a total distance close to 100km. If they score 0 goals, distance covered doesn’t matter. Can you see this by tracing through the decision tree with 0 goals?

But distance can impact predictions if they score goals. Make sure you can see this from the 2D partial dependence plot. Can you see this pattern in the decision tree too?

图形展示了所有的得分与跑动距离的组合。

比如说我们看到最有可能得mvp的是至少进一个球并且跑步距离接近100km。如果他们得0分的话, 那么跑动距离将没有意义。你能在决策树上去搜索得分0吗？

一旦他们得分了，那么跑动距离将会影响结果。在2维的部分特征图上你能看到这些，你是否也能在决策树上找到这个呢？

https://www.kaggle.com/dansbecker/partial-plots

Intro 简介

One of the most basic questions we might ask of a model is What features have the biggest impact on predictions?

在数据模型中有一个很基础的问题：哪一个特征对预测结果影响最大。

This concept is called feature importance. I’ve seen feature importance used effectively many times for every purpose in the list of use cases above.

这种概念被叫做特征重要性（我觉得跟特征选择或者特征工程这一系列的概念有联系。）根据以往经验，特征重要性这个概念被很有效的利用在跟中不同的项目上。

There are multiple ways to measure feature importance. Some approaches answer subtly different versions of the question above. Other approaches have documented shortcomings.

有许多种可以测量特征重要性的方法，有一些方法对问题有不同的解释方法，其他的方法有明显的缺点。（总而言之没有一种万能的方法）

In this lesson, we’ll focus on permutation importance. Compared to most other approaches, permutation importance is:

在这门课中，我们会介绍交换排列计算重要性的方法。对比其他方法，交换排列计算重要性可以

Fast to calculate

更快计算

Widely used and understood

应用更普遍

Consistent with properties we would want a feature importance measure to have

保持特征重要性测量的一致性

How it Works 应用

Permutation importance uses models differently than anything you’ve seen so far, and many people find it confusing at first. So we’ll start with an example to make it more concrete.

交换排列计算重要性和我们之前见到的用model的方法不一样，一开始很多人都会觉得这个很难懂。所以我们会举一个例子让这个更具体，更容易让人接受。

Consider data with the following format:

We want to predict a person’s height when they become 20 years old, using data that is available at age 10.

我们想用这些人在十岁的数据来越策他们20岁的身高。

Our data includes useful features (height at age 10), features with little predictive power (socks owned), as well as some other features we won’t focus on in this explanation.

我们的数据包括很多很有用的特征比如10岁的身高，以及一些很弱的特征，比如他们拥有多少袜子，以及其他，我们并不详述。

Permutation importance is calculated after a model has been fitted. So we won’t change the model or change what predictions we’d get for a given value of height, sock-count, etc.

交换排列计算重要性在训练数据之后被计算。所以我们并不会更改数据或是更改预测结果。

Instead we will ask the following question: If I randomly shuffle a single column of the validation data, leaving the target and all other columns in place, how would that affect the accuracy of predictions in that now-shuffled data?

相反我们会问以下的问题，如果我让一列随机排列这些预测数据，然后让其他的数据，特征量都不变，然后比较与原结果有多少准确度上的不同。

Randomly re-ordering a single column should cause less accurate predictions, since the resulting data no longer corresponds to anything observed in the real world. Model accuracy especially suffers if we shuffle a column that the model relied on heavily for predictions. In this case, shuffling height at age 10 would cause terrible predictions. If we shuffled socks owned instead, the resulting predictions wouldn’t suffer nearly as much.

当我们随机重新排列期中一列上的数据，往往会带来更低准确度的预测，因为这个数据和真实世界并不符合。准确度会根据重新排列的那一行对结果影响的权重而降低不同级别。于是我们知道如果重新排列10岁身高哪一行会带来更差的结果预测。如果我们重新排列拥有袜子数量那一列，并不会对结果有多少影响。

With this insight, the process is as follows:

根据我们的观察，过程应该是

1. Get a trained model

得到一个训练好的模型

2. Shuffle the values in a single column, make predictions using the resulting dataset. Use these predictions and the true target values to calculate how much the loss function suffered from shuffling. That performance deterioration measures the importance of the variable you just shuffled.

随机排列单独一列的数据，然后来预测结果。通过损失函数来计算有多接近原来的结果。根据我们之前所推论的比较原有结果，表现越差的就代表这个特征对结果正面影响越大。

3. Return the data to the original order (undoing the shuffle from step 2.) Now repeat step 2 with the next column in the dataset, until you have calculated the importance of each column.

然后还原数据，重复第2步直到我们计算出所有特征的结果影响的重要性。

Code Example

Our example will use a model that predicts whether a soccer/football team will have the “Man of the Game” winner based on the team’s statistics. The “Man of the Game” award is given to the best player in the game. Model-building isn’t our current focus, so the cell below loads the data and builds a rudimentary model.

关于预测足球队里谁能得到足球先生的预测。

1
2
3
4
5
6
7
8
9
10
11
12

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier

data = pd.read_csv('../input/fifa-2018-match-statistics/FIFA 2018 Statistics.csv')
y = (data['Man of the Match'] == "Yes")  # Convert from string "Yes"/"No" to binary
feature_names = [i for i in data.columns if data[i].dtype in [np.int64]]
X = data[feature_names]
train_X, val_X, train_y, val_y = train_test_split(X, y, random_state=1)
my_model = RandomForestClassifier(random_state=0).fit(train_X, train_y)

Here is how to calculate and show importances with the eli5 library:

1
2
3
4
5

import eli5
from eli5.sklearn import PermutationImportance

perm = PermutationImportance(my_model, random_state=1).fit(val_X, val_y)
eli5.show_weights(perm, feature_names = val_X.columns.tolist())

Interpreting Permutation Importances 解释交换排列计算重要性

The values towards the top are the most important features, and those towards the bottom matter least.

在这张图中 最上面的是最重要的特征，反之最下面的是最不重要的特征。

The first number in each row shows how much model performance decreased with a random shuffling (in this case, using “accuracy” as the performance metric).

每一行中的第一个数字显示在一个随机排列之后整个模型的准确度下降了多少。

Like most things in data science, there is some randomness to the exact performance change from a shuffling a column. We measure the amount of randomness in our permutation importance calculation by repeating the process with multiple shuffles. The number after the ± measures how performance varied from one-reshuffling to the next.

和许多在数据科学中发生的事情类似，有很多随机的事情会在随机排列一个特征列的时候发生，我们可以增加随机性通过更多次的随机排列。毕竟每次的随机排列都不会得到同样的结果。

You’ll occasionally see negative values for permutation importances. In those cases, the predictions on the shuffled (or noisy) data happened to be more accurate than the real data. This happens when the feature didn’t matter (should have had an importance close to 0), but random chance caused the predictions on shuffled data to be more accurate. This is more common with small datasets, like the one in this example, because there is more room for luck/chance.

有时候你甚至很偶然的可以得到一些负值（意思是随机排列那一列的时候准确度反而上升了）。这种情况往往是因为这个被随机排列这里一列本身就是一些无用的数据。当你用更大的数据去训练，这种幸运也会越来越难发生。

In our example, the most important feature was Goals scored. That seems sensible. Soccer fans may have some intuition about whether the orderings of other variables are surprising or not.

在我们的例子中，最重要的特征是进球得分。这非常合理，球迷还是很理性的会把进球数当作选足球先生第一个的参考值。

Reference

https://www.kaggle.com/dansbecker/permutation-importance

Use Cases for Model Insights

在data sciense 里用到的直觉

• Debugging

• Information Feature engineering

• Directing future data collection

• Informing human decision-making

• Building Trust 建立信任

Debugging

The world has a lot of unreliable, disorganized and generally dirty data. You add a potential source of errors as you write preprocessing code. Add in the potential for target leakage and it is the norm rather than the exception to have errors at some point in a real data science projects.

这个世界有许多不现实的，没有很好组织的，普遍的脏数据。当你对代码进行预处理的时候，你要去做一些规则去预防数据上潜在的错误，以及Data Leakage（Data Leakage一些不能用的数据，但是强行用了，使因果关系颠倒）。在真实的数据科学项目上，我们都应该提前制定规则去预防而不是在训练中或者结束用一些exception去去除。

Given the frequency and potentially disastrous consequences of bugs, debugging is one of the most valuable skills in data science. Understanding the patterns a model is finding will help you identify when those are at odds with your knowledge of the real world, and this is typically the first step in tracking down bugs.

考虑到这些数据bug经常会引起一系列的灾难，debugging 在数据科学中是最有价值的技能之一。你需要理解这个数据模型以及一部分真实世界的常识去作为定位bug的第一步

Informing Feature Engineering

Feature engineering is usually the most effective way to improve model accuracy. Feature engineering usually involves repeatedly creating new features using transformations of your raw data or features you have previously created.

特征上的分析运算往往是加强数据模型最有效的方法。Feature Engineering包括重复不停去利用这些特征做一些变化组合来得到新的特征

Sometimes you can go through this process using nothing but intuition about the underlying topic. But you’ll need more direction when you have 100s of raw features or when you lack background knowledge about the topic you are working on.

有时候你只要用你对某一领域或者课题的直觉，但是当你缺少这个领域的知识的时候且有很多特征量，你需要试验更多的方向。

A Kaggle competition to predict loan defaults gives an extreme example. This competition had 100s of raw features. For privacy reasons, the features had names like f1, f2, f3 rather than common English names. This simulated a scenario where you have little intuition about the raw data.

有一个kaggle 的竞赛去预测loan defaults（默认贷款之类的），给了一个极端的例子。这次竞赛有一百个原始特征，但是由于一些隐私问题，所有的特征都被命名为f1,f2,f3, 而不是常规的特征名字。这个时候就模拟了一种你并不拥有原始数据的背景知识的情况。

One competitor found that the difference between two of the features, specificallyf527 - f528, created a very powerful new feature. Models including that difference as a feature were far better than models without it. But how might you think of creating this variable when you start with hundreds of variables?

有一个参赛者发现有两个特征的差，尤其是f527 - f528，这个差值或者创造的新的特征对结果的影响非常大。但是这也只能说是一个巧合当你需要去分析成百上千的特征

The techniques you’ll learn in this course would make it transparent that f527 and f528 are important features, and that their role is tightly entangled. This will direct you to consider transformations of these two variables, and likely find the “golden feature” of f527 - f528.

你从这门课程学到的技巧可以更轻易的分辨出f527 和 f528 是非常重要的特征。并且他们的角色是互相纠缠影响的。这个方向就有利于我们去思考怎么对这两个特征进行变形，然后有可能我们就可以发现那种极品的特征 比如f527 - f528

As an increasing number of datasets start with 100s or 1000s of raw features, this approach is becoming increasingly important.

随着特征量的增加从100 到1000 数量级，这个方法变得尤为重要。

Directing Future Data Collection

对收集新数据的指向

You have no control over datasets you download online. But many businesses and organizations using data science have opportunities to expand what types of data they collect. Collecting new types of data can be expensive or inconvenient, so they only want to do this if they know it will be worthwhile. Model-based insights give you a good understanding of the value of features you currently have, which will help you reason about what new values may be most helpful.

我们并不能控制我们从网上下载的数据库。但是很多公司或者一些机构会利用数据科学找机会去拓展一部分新数据，当然刚开始获取新数据的代价是非常高的。所以他们往往需要模型结构的帮助去分辨获取新数据是否值得，以及新的数据能给整个项目带来多大的变化。

Informing Human Decision-Making

一些人类的决定

Some decisions are made automatically by models. Amazon doesn’t have humans (or elves) scurry to decide what to show you whenever you go to their website. But many important decisions are made by humans. For these decisions, insights can be more valuable than predictions.

模型会自动做出一些决定。Amazon 并没有神仙来时时刻刻决定你到Amazon网站看到了神马。但是很多重要的决定还是由人类决定，直觉有的时候比机器预测做出更有价值的决定。

Building Trust

建立信任

Many people won’t assume they can trust your model for important decisions without verifying some basic facts. This is a smart precaution given the frequency of data errors. In practice, showing insights that fit their general understanding of the problem will help build trust, even among people with little deep knowledge of data science.

许多人并不依靠我们的模型去做一些重要的决定，如果我们的模型没办法符合很基础的事实。人类本能就会预防一些未知的错误，包括数据的错误。在测试中，我们会用直觉去理解这个问题，让整个数据结构变得合理并却让一些没有数据科学训练的人也容易接受。

Reference:

https://www.kaggle.com/dansbecker/use-cases-for-model-insights

人類發展史與外星文明

經過2019 一月份的加拿大天文臺收到疑爲外星文明信號事件，和朋友聊了一下關於爲什麽人類這樣高智慧型的文明存在的最重要的元素。

由於我并不是個文科生，也沒有豐富的人類學，歷史學等積澱與素養。我的觀點有點奇葩，我覺得就是幸運或者是運氣。可能部分符合大過濾器的一些猜想。

• 1. 太陽這顆恆星非常的穩定，并卻地球在太陽系的位置非常有利於液態水的存在。原諒我是一個三體迷，我深深得同意大劉，地球的確是一個得天獨厚的天堂這一觀點。稀有地球理論
• 2. 接下來這一點比較奇葩，也是我不太贊同大劉的所有觀點的地方。

如果按照大劉的兩條邏輯鏈：技術爆炸和黑暗森林法則。那麽我覺得完全按照這兩個邏輯生長的文明社會反而不會擁有最强大的科技。也是被金庸的想法所影響，在天龍八部裏，鳩摩智用小無相功驅動少林七十二絕技。但是按照掃地僧所説的每一種佛法都對應一種絕技，如果只練絕技而不修行佛法就會傷及自身。

愚以爲這是一種比較大的智慧，如果把這個思想對比到人類社會思想文明和科技水平。如果只是針對科技發展，而不等待社會文明的進步。那麽這個生物文明就會陷入一種岌岌可危自我毀滅的可能性。

我知道這個比較難以闡述，比如説人人都有可能造出毀滅地球的一種武器，無論是核武器還是生化武器，如果思想道德還沒有到達那種水平比如西方的政治正確人人平等或者是東方孔子的己所不欲勿施於人。那麽一些極端主義就容易生成那種老子活不好你們也別想活的地步。他們手裏又非常容易擁有那種高級的科技。我本人非常相信這種可能性。如同現在小部分的恐怖分子又或者極端主義

儅大劉的邏輯鏈成立的時候，他是把文明當成個體,但是如果換成每個人都是黑暗森林法則中的個體的時候，科技文明就只剩下隱藏自己，以及儅森林獵人這兩種。

所以我認爲人類的幸運在於我們的道德文明，又或者社會文明走在了科技文明的前面，我們并沒有2000年前大規模使用電器，1000年前發展出核武器。并沒有讓一個部落文明或者奴隸制社會擁有大規模殺傷性武器。

當然我朋友也跟我爭論，他認爲社會文明，道德文明是科技文明發展的基礎。我覺得不完全正確，比如歐洲當時很多貴族在容易滿足生理需求的前提下，也經常去研究數學以及其他科學。甚至我認爲即使是奴隸制的社會架構下也是可以誕生高科技產物的。但是這個保留有意見。

可以預想如果成千上萬的奴隸主都掌握核心技術，和強破壞性武器，按概率學上來説，只要有一個發瘋想制裁別人，那麽這個文明就亡了。

Docker 有三个基本概念：

• 镜像 Image

• 容器 Container

• 仓库 Repository

镜像

Docker 镜像（Image）就相当于是一个 root 文件系统。Docker 镜像 是一个特殊的文件系统，除了提供容器运行时所需的程序、库、资源、配置等文件外，还包含了一些为运行时准备的一些配置参数（如匿名卷、环境变量、用户等）。镜像不包含任何动态数据，其内容在构建之后也不会被改变。

分层存储

镜像只是一个虚拟的概念，其实际体现并非由一个文件组成，而是由一组文件系统组成，或者说，由多层文件系统联合组成。

镜像构建时，会一层层构建，前一层是后一层的基础。每一层构建完就不会再发生改变，后一层上的任何改变只发生在自己这一层。比如，删除前一层文件的操作，实际不是真的删除前一层的文件，而是仅在当前层标记为该文件已删除。在最终容器运行的时候，虽然不会看到这个文件，但是实际上该文件会一直跟随镜像。因此，在构建镜像的时候，需要额外小心，每一层尽量只包含该层需要添加的东西，任何额外的东西应该在该层构建结束前清理掉。

分层存储的特征还使得镜像的复用、定制变的更为容易。甚至可以用之前构建好的镜像作为基础层，然后进一步添加新的层，以定制自己所需的内容，构建新的镜像。

容器

https://yeasy.gitbooks.io/docker_practice

PCA的数学原理

PCA（Principal Component Analysis）是一种常用的数据分析方法(降维)

1. 数据的向量表示及降维问题

我们要在降维的同时让数据信息资源的损失尽可能降低.

朴素降维思考，比如男女column， M F，可以去掉一列，因为非黑即白。

eg: 数据记录为

(日期, 浏览量, 访客数, 下单数, 成交数, 成交金额) => （500，240，25，13，2312.15）T

比如浏览量和访客数是有关联的，可以简单降维

2. 向量的表示及基变换

3. 内积与投影

内积, 高中学的叫点乘，或者还看见有人叫点积（Dot Product）

在几何定义为向量A 和 向量B的长度 乘以 cos（夹角）如果B的模长度为1，那么AB 的点乘就是A 投影在B上的长度

4. 基 basis

原理上是在x轴上的投影为3，在y轴上的投影为2。 从代数原理上，（3，2） = 3x（1，0）+2x（0，1）

（1，0） 和 （0，1） 就是二维空间中的一组基，基的模长度往往为1

我们可以变换这组基的方向，比如（1，1）， （-1，1），然后把他转化为基，即为 (\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})和(-\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})。

基变换 （二维）

顺时针 theta 角度

cos(theta) -sin(theta)
sin(theta) cos(theta)

5. 基变换的矩阵表示

原坐标 [1 0], [0 1]

[1 0] [3] => [3]
[0 1] [2] [2]

新坐标

当把M个N维向量变换到R个N维向量表示的新空间中去，数学表达式为

R是可以小于N的，R是基的数量，可以讲N维数据变换到更低的维度中去。这种矩阵相乘表示为降维变换。

6. 协方差矩阵及优化目标

关于如何选择最优的基，比如从N维向量降维到K。

例子 把五条2维向量 降维到 1维空间

(1,1) (1,3) (2,3) (4,4) (2,4)

先减去均值

(-1,-2) (-1,0) (0,0) (2,1) (0,1)

如果选一条线当一维坐标，如果选X轴(0,1) (0,0) 重叠， 如果选y轴(-1,0), (0,0) 重叠

我的理解是应该选一种空间 让每个投影都尽量分散

7. 方差

上面提到的问题变成寻找一个一维基上投影的点方差最大

8. 协方差

如果降到二维，还是坚持最大方差的话那么第二条线会与第一条线重合，但是最理想情况是第二条线和第一条线独立。

协方差

当协方差为0的时候两个向量完全独立，也就是正交（是垂直吗？）

至此，我们得到了降维问题的优化目标：将一组N维向量降为K维（K大于0，小于N），其目标是选择K个单位（模为1）正交基，使得原始数据变换到这组基上后，各字段两两间协方差为0，而字段的方差则尽可能大（在正交的约束下，取最大的K个方差）。

9. 协方差矩阵

（数学真是太有意思了）

假设我们有两个字段a,b, 我们把它们按行组成矩阵 X

然后自己相乘并乘上系数

结果大概是
[a方差， 协方差]
[协方差, b方差]

10. 协方差矩阵对角化

于是根据上面的推导，我们要让对角线上的数据尽可能的高，因为方差变大，然后非对角线的协方差尽可能的小，因为每个基要独立。这样就达成了优化的条件。

接下来有点难懂：我先把每个数据都列下来

m为字段数(我的理解是需要投影的点)

n为基的维度 (我的理解是原数据的维度：可能错)

X 为基组成的矩阵 m x n

C 为一个对称矩阵

P 为行向量基组成的矩阵 r x m (r是什么：r可能是下降后的基的数目)

Y = PX Y是P投影在空间X上，X对P做的基变换

要找到P，满足 是一个对角矩阵

对角元素从大到小排列，降到K维就取K行

实对称矩阵

协方差矩阵 C 是一个实对成矩阵(还未消化)

1 - 实对称矩阵不同特征值对应的特征向量必然正交。

2 - 设特征向量lambda重数为r，则必然存在r个线性无关的特征向量对应于lambda，因此可以将这r个特征向量单位正交化。

P是协方差矩阵的特征向量单位化后按行排列出的矩阵，其中每一行都是C的一个特征向量,

P按照D中特征值的从大到小排列，P的前K行组成的矩阵乘以原始数据矩阵X，就得到了我们需要降维后的矩阵Y

11. 算法及实例

总结一下PCA的算法步骤：

设有m条n维数据。

1）将原始数据按列组成n行m列矩阵X

2）将X的每一行（代表一个属性字段）进行零均值化，即减去这一行的均值

3）

4）求出协方差矩阵的特征值及对应的特征向量 特征值向量求法

5）将特征向量按对应特征值大小从上到下按行排列成矩阵，取前k行组成矩阵P

6）Y=PX即为降维到k维后的数据

举例那个去reference里面看更清楚，准备翻译成 IPython 更清晰解释PCA 降维

Reference

知乎PCA的数学原理 貌似也是转的，找不到原link

神经网络笔记2

设置数据和模型

神经网络就是进行了一系列的线性映射与非线性激活函数交织的运算, 这些做法共同定义了评分函数(score function)

数据预处理

3个常用的符号，数据矩阵X，假设其尺寸是[N x D]（N是数据样本的数量，D是数据的维度）

均值减法(Mean subtraction):

每个独立特征减去平均值，从几何上可以理解为在每个维度上都将数据云的中心都迁移到原点.

numpy代码：

1

X -= np.mean(X, axis=0)

归一化（Normalization）

数据的所有维度都归一化，使其数值范围都近似相等.

有两种方法：

第一种是先对数据做零中心化,然后每个维度都除以其标准差

numpy 代码

1
2

X -= np.mean(X, axis=0)
X /= np.std(X, axis=0)

第二种方法是对每个维度都做归一化，每个维度最大值1，最小值-1

PCA和白化（Whitening）To learning

1
2
3

# 假设输入数据矩阵X的尺寸为[N x D]
X -= np.mean(X, axis = 0) # 对数据进行零中心化(重要)
cov = np.dot(X.T, X) / X.shape[0] # 得到数据的协方差矩阵

Reference:

神经网络笔记 2