<jupyter_start><jupyter_text>评价函数模块 `_criterion.*` 详解=================================  0. 大纲本文会按照类的继承关系，由上到下介绍各个类的成员及实现细节，尽量说得详细点。<jupyter_code>SVG("./res/uml/Model___criterion_1.svg")<jupyter_output><empty_output><jupyter_text>1. Criterion, 接口类Criterion 作为接口之用，相当于 Java 中的抽象类。其接口定义在 `_criterion.pxd` 文件，实现在 `_criterion.pyx` 中。我们对 Criterion 分为两部份来说明：内部成员，和函数方法。 1.0 内部成员为了计算和存储效率，sklearn 做了些针对性的数据设计。当然，其实思路挺直觉的，我们带着具体的成员变量来说明。+ 数据分条 (stride)  二维数组会用一维数组（其实是个连续的缓存）加上偏移量来实现，我以前在数字图像处理中常看到。          ```Python     26      Internal structures     27     cdef DOUBLE_t* y                      Values of y     28     cdef SIZE_t y_stride                  Stride in y (since n_outputs >= 1)     29     cdef DOUBLE_t* sample_weight          Sample weights  ```  分类的标签或者回归的值（$y$）可能是个向量，所以需要对 $y$ 值分条，每条的长度自然是 `y_stride`。       `sample_weight` 是各个样本的权重值，不用多讲。 + 使用索引  决策树分割节点时会涉及到移动样本，因为 $x$ 通常维数较多，开销较大。对于这类问题，一个常见的做法就是交换索引，而不变动值。具体到数组操组，这个索引就是下标值。    ```Python  31     cdef SIZE_t* samples                  Sample indices in X, y  32     cdef SIZE_t start                     samples[start:pos] are the samples in the left node  33     cdef SIZE_t pos                       samples[pos:end] are the samples in the right node  34     cdef SIZE_t end  ```    这里样本值的索引区间其实是前闭后开，即 samples[start:pos) 和 samples[pos:end)。 + 加权处理  加权的样本数目统计  ```Python  36     cdef SIZE_t n_outputs                 Number of outputs  37     cdef SIZE_t n_node_samples            Number of samples in the node (end-start)  38     cdef double weighted_n_samples        Weighted number of samples (in total)  39     cdef double weighted_n_node_samples   Weighted number of samples in the node  40     cdef double weighted_n_left           Weighted number of samples in the left node  41     cdef double weighted_n_right          Weighted number of samples in the right node  ```    加权的 $y$ 值累加和（若是分类，则是对各个类别单独做累加）  ```Python  43     cdef double* sum_total           For classification criteria, the sum of the  44                                      weighted count of each label. For regression,  45                                      the sum of w*y. sum_total[k] is equal to  46                                      sum_{i=start}^{end-1} w[samples[i]]*y[samples[i], k],  47                                      where k is output index.  48     cdef double* sum_left            Same as above, but for the left side of the split  49     cdef double* sum_right           same as above, but for the right side of the split  ```  1.1 函数方法+ 纯接口```Python 54      Methods 55     cdef void init(self, DOUBLE_t* y, SIZE_t y_stride, DOUBLE_t* sample_weight, 56                    double weighted_n_samples, SIZE_t* samples, SIZE_t start, 57                    SIZE_t end) nogil 58     cdef void reset(self) nogil 59     cdef void reverse_reset(self) nogil 60     cdef void update(self, SIZE_t new_pos) nogil 61     cdef double node_impurity(self) nogil 62     cdef void children_impurity(self, double* impurity_left, 63                                 double* impurity_right) nogil 64     cdef void node_value(self, double* dest) nogil``` + 计算不纯度提升量```Python 65     cdef double impurity_improvement(self, double impurity) nogil 66     cdef double proxy_impurity_improvement(self) nogil````proxy_impurity_improvement`：\begin{align}    & \frac{N_t}{N} \left ( \text{ impurity } - \frac{N_{t,R}}{N_t} \text{ right_impurity } - \frac{N_{t,L}}{N_t} \text{ left_impurity } \right ) \\    &= \frac{N_t}{N} \text{ impurity } - \frac{N_{t,R}}{N} \text{ right_impurity } - \frac{N_{t,L}}{N} \text{ left_impurity } \\    &= \frac{N_t}{N} \text{ impurity } - \frac{1}{N} \underbrace{ \left ( N_{t,R} \text{ right_impurity } - N_{t,L} \text{ left_impurity } \right ) }_{\text{impurity_improvement}} \\\end{align}对一个节点，去除公式中的常量，弄了个 `impurity_improvement` 来减少计算量。  2. 分类评价函数  2.0 ClassificationCriterionClassificationCriterion 类是分类评估函数的父类，主要𫐄实现初始化和一些预处理函数。我们从上到下依序看代码。 2.0.0 变量```Python 207 cdef class ClassificationCriterion(Criterion): 208     """Abstract criterion for classification.""" 209 210     cdef SIZE_t* n_classes 211     cdef SIZE_t sum_stride```这里分类为了支持多输出（target）问题，用 `n_classes` 来统计每个输出的类别个数，用 `sum_stride` 来对齐不同输出的结果。用文字表述比较困难，下面的图说明更清晰点。<jupyter_code>show_image("./res/ClassificationCriterion.jpg", figsize=(8,15))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10236768802228412, 'fraction_numerical': 0.023676880222841225, 'mean_word_length': 2.9495875343721356, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '295241', 'license': 'CC0-1.0', 'n_tokens_mistral': 1833, 'n_tokens_neox': 1811, 'n_words': 384, 'path': 'machine_learning/tree/decision_tree/sklearn/_criterion.ipynb', 'repo_name': 'ningchi/book_notes', 'size': '4308'}