<jupyter_start><jupyter_text>数据以及参数<jupyter_code>random_seed = 42<jupyter_output><empty_output><jupyter_text>其余基本模型的参数都是sklearn默认的设置深度森林参数：最大层数20，连续5层没有效果（WF1）提升则停止，每一层：4个随机森林，3个决策树，1个逻辑回归<jupyter_code>def get_toy_config():
    config = {}
    ca_config = {}
    ca_config["random_state"] = random_seed
    ca_config["max_layers"] = 20
    ca_config["early_stopping_rounds"] = 5
    ca_config["n_classes"] = 6
    ca_config["estimators"] = []
    ca_config["estimators"].append({"n_folds": 5, "type": "RandomForestClassifier", "random_state" : random_seed})
    ca_config["estimators"].append({"n_folds": 5, "type": "RandomForestClassifier", "random_state" : random_seed})
    ca_config["estimators"].append({"n_folds": 5, "type": "RandomForestClassifier", "random_state" : random_seed})
    ca_config["estimators"].append({"n_folds": 5, "type": "RandomForestClassifier", "random_state" : random_seed})
    ca_config["estimators"].append({"n_folds": 5, "type": "DecisionTreeClassifier"})
    ca_config["estimators"].append({"n_folds": 5, "type": "DecisionTreeClassifier"})
    ca_config["estimators"].append({"n_folds": 5, "type": "DecisionTreeClassifier"})
    ca_config["estimators"].append({"n_folds": 5, "type": "LogisticRegression"})
    config["cascade"] = ca_config
    return config
path = os.getcwd()+'/../data/20122018freshwater_four_feature.csv'
data = pd.read_csv(path, na_values = np.nan)
# training/valid/test: 0.6/0.2/0.2, 各数据集划分的时候要注意
X = data.drop(['本周水质'], axis=1).values # Series
y = data['本周水质'].values-1 # 转换标记为0开始，因为argmax()的范围为0-n
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, 
                                       stratify = y, random_state = random_seed)
# Z-score
clean_pipeline = Pipeline([('imputer', preprocessing.Imputer(missing_values='NaN',strategy="median")),
                           ('std_scaler', preprocessing.StandardScaler()),])
X_train = clean_pipeline.fit_transform(X_train)
X_test = clean_pipeline.fit_transform(X_test)
X_train2, X_valid, y_train2, y_valid = train_test_split(X_train, y_train, test_size=0.25, 
                                       stratify = y_train, random_state = random_seed)
X_train = X_train2
y_train = y_train2<jupyter_output><empty_output><jupyter_text>Base Model Pre-train<jupyter_code>config = get_toy_config()

# models = [
#     LogisticRegression(),
#     LinearDiscriminantAnalysis(),
#     SVC(probability=True),
#     DecisionTreeClassifier(),
#     ExtraTreeClassifier(),
#     GaussianNB(),
#     KNeighborsClassifier(),
#     RandomForestClassifier(random_state=random_seed),
#     ExtraTreesClassifier(random_state=random_seed),
#     GCForest(config)
# ]


# 除去所有表现优异的树模型，保留3个基础的机器学习模型
models = [
    LogisticRegression(),
    LinearDiscriminantAnalysis(),
    SVC(probability=True),
    GaussianNB(),
    KNeighborsClassifier(),
]
y_pred_proba_all = [] # 各模型权重参数

test_entries = [] # 各基模型测试集结果
train_entries = [] # 各基模型训练集结果

for model in models:
    model_name = model.__class__.__name__
    if model_name == 'GCForest':
        model.fit_transform(X_train, y_train, X_test, y_test)
    else:
        model.fit(X_train, y_train)

    y_train_pred = model.predict(X_train)
    y_test_pred = model.predict(X_test)
    y_pred_proba = model.predict_proba(X_valid) # 20%验证集上的概率向量，为训练权重做准备
    
    f1_train = f1_score(y_train, y_train_pred, average='weighted')
    f1_test = f1_score(y_test, y_test_pred, average='weighted')
    acc_train = accuracy_score(y_train, y_train_pred)
    acc_test = accuracy_score(y_test, y_test_pred)

    train_entries.append((model_name, f1_train, acc_train))
    test_entries.append((model_name, f1_test, acc_test))
    y_pred_proba_all.append(y_pred_proba)
    
train_df = pd.DataFrame(train_entries, columns=['model_name', 'train_f1_weighted', 'train_accuracy'])
test_df = pd.DataFrame(test_entries, columns=['model_name', 'test_f1_weighted', 'test_accuracy'])
print("Results on training data")
train_df
print("Results on test data")
test_df<jupyter_output>Results on test data<jupyter_text>训练权重<jupyter_code>y_pred_proba_all = np.asarray(y_pred_proba_all)
np.save("../npy/NCE_proba_of_5models.npy", y_pred_proba_all)
# 直接载入各个模型在验证集的概率向量，仅在懒得pre train的时候使用
# y_pred_proba_all = np.load("../npy/NCE_proba_of_10models.npy")<jupyter_output><empty_output><jupyter_text>Use NCE function to get the weights<jupyter_code>def trainNCE(y_valid, y_pred_proba_all, model_num):
    classifier_num = model_num
    population_num = 1000
    retain_population_num = 100
    max_iteration = 50
    population_weights = np.zeros((population_num, classifier_num))
    population_retain_weights = np.zeros((retain_population_num, classifier_num))
    population_score = []
    population_retain_score = []

    all_best_weights = np.zeros((max_iteration, classifier_num)) # 某次训练时，所有迭代步骤中最好的种群的权重
    all_best_f1s = np.zeros(max_iteration) # 某次训练时，每次迭代都取精英种群中最高的f1，构成这个“最高f1数组”
    all_mean_f1s = np.zeros(max_iteration) # 某次训练时，每次迭代都取精英种群f1的均值，构成这个“平均f1数组”

    mu = np.zeros(classifier_num)
    sigma = np.ones(classifier_num)


    # 在验证集集上: 训练每个基学习器的投票参数
    for i in range(max_iteration):
        print("Iteration: %d" %(i))
        # 该次迭代的所有种群们
        population_score = np.zeros(population_num)
        population_weights = np.zeros((population_num, classifier_num))
        # 该次迭代的优势种群们
        population_retain_score = np.zeros(retain_population_num)
        population_retain_weights = np.zeros((retain_population_num, classifier_num))

        # 生成所有种群
        for j in range(classifier_num):
            w = np.random.normal(mu[j], sigma[j]+700/(i+1), population_num)
#             w = np.random.normal(mu[j], sigma[j], population_num)
            population_weights[:,j] = w

        # 映射所有种群的权重至[0:1]
        for j in range(population_num):
            w2 = np.zeros(classifier_num)
            for k in range(classifier_num):
                w2[k] = np.exp(-population_weights[j][k]*population_weights[j][k])
#                 w2[k] = np.exp(population_weights[j][k])/np.sum(np.exp(population_weights[j]))
            population_weights[j] = w2

        # 计算所有种群得分
        for j in range(population_num):
            y_pred_ensemble_proba = np.zeros((len(y_valid), 6)) # 集成器概率向量
            # 为每一个基学习器乘上权重
            for k in range(classifier_num):
                y_pred_ensemble_proba += y_pred_proba_all[k] * population_weights[j][k]
            y_pred_ensemble = np.argmax(y_pred_ensemble_proba, axis=1)
#             print("esemble:",y_pred_ensemble)
#             print("true",y_valid)
            f1 = f1_score(y_valid, y_pred_ensemble, average="weighted")
            population_score[j] = f1

        # 所有种群得分按降序排列
        retain_index = np.argsort(-np.array(population_score))[:retain_population_num]

        # 记录该次迭代中的优势种群们
        population_retain_weights = population_weights[retain_index] # 精英样本权重
        population_retain_score = np.array(population_score)[retain_index] # 精英样本得分

        # 记录每次迭代最好的种群和value
        all_best_weights[i] = population_retain_weights[0]
        all_best_f1s[i] = population_retain_score[0] # 最高得分
        all_mean_f1s[i] = np.mean(population_retain_score) # 精英样本平均得分
        # 更新mu，sigma为优势种群们的分布
        mu = np.mean(population_retain_weights, axis = 0)
        sigma = np.std(population_retain_weights, axis = 0) #default: ddof = 0, The divisor used in calculations is N - ddof
        # print("mu\n",mu)
        # print("sigma\n", sigma)
        # print("Weighted F1 Score after rank")
        # print(population_retain_score)
        # print("Weights")
        # print(population_retain_weights)
    
    # 权重训练完毕
    print("==================Finish training==================")
    last_weight = population_retain_weights[0] # 最后一轮迭代精英样本中具有最高F1的权重
    last_f1 = all_best_f1s[-1] # 最后一轮迭代中精英样本中的最高F1
    best_f1 = all_best_f1s[np.argmax(all_best_f1s)] # 所有轮迭代中最高的F1
    best_weight = all_best_weights[np.argmax(all_best_f1s)] # 所有轮迭代中具有最高F1
    
    print("Best F1 in last iteration: %f" % (last_f1))
    print("Last weight in last iteration: %s" %(last_weight))

    print("Best F1 in all iterations: %f" % (best_f1))
    print("Best weight in all iterations: %s" %(best_weight))
    
    print("Last mu\n", mu)
    print("Last sigma\n", sigma)
    
    plt.figure(figsize=(10,6))
    plt.plot(all_best_f1s[:25], 'b', label = 'Best weighted F1 score of elite samples')
    plt.plot(all_mean_f1s[:25], 'r', label = 'Mean weighted F1 score of elite samples')
    plt.xlabel('Iteration')
    plt.ylabel('Weighted F1 score')
    plt.legend(frameon=False)
    plt.grid(True)
    # plt.savefig('../img/weighed_F1_iteration(1-10).eps',format='eps')
    return last_weight<jupyter_output><empty_output><jupyter_text>Use CMAES function to get the weights<jupyter_code>def trainCMAES(y_valid, y_pred_proba_all, model_num):
    es = cma.CMAEvolutionStrategy(model_num * [0], 0.5)
#     help(cma)
#     help(es)
    score = "f1_weighted"
#     score = "accuracy"
    while not es.stop():
        solutions = es.ask()
        es.tell(solutions, [cma.ff.water_ensemble(x, y_pred_proba_all, y_valid, metric=score) for x in solutions])
        es.logger.add()  # write data to disc to be plotted
        es.disp()
    # 训练完毕
    es.result_pretty()  # pretty print result
    cma.plot()    
    weights = np.exp(es.result.xbest)/np.sum(np.exp(es.result.xbest))
    return weights<jupyter_output><empty_output><jupyter_text>训练基模型的权重<jupyter_code>NCE_weights = trainNCE(y_valid, y_pred_proba_all, model_num=5)
CMAES_weights = trainCMAES(y_valid, y_pred_proba_all, model_num=5)
np.save("../npy/NCE_weights_5.npy", NCE_weights)
np.save("../npy/CMAES_weights_5.npy", CMAES_weights)<jupyter_output><empty_output><jupyter_text>测试 代入权重，在3个集合上计算ACC和F1- 训练集（总样本数量的60%）X_train- 验证集（总样本数量的20%，调参数，用CMAES得到权重）X_valid- 测试集（总样本数的20%）X_test- 各模型稳定性基于交叉验证，在除测试集外的80%上做5折 X_train<jupyter_code>print("==================== NCE weights ====================")
print(NCE_weights)
print("==================== CMAES weights ====================")
print(CMAES_weights)<jupyter_output>==================== NCE weights ====================
[5.30099214e-16 4.13222677e-97 2.29547031e-05 1.41943724e-22
 2.32604143e-05]
==================== CMAES weights ====================
[0.00255831 0.00118493 0.38039359 0.20452462 0.41133857]<jupyter_text>ACC<jupyter_code># 取训练好的模型
for model in models:
    model_name = model.__class__.__name__
    train_pred = model.predict(X_train)
    valid_pred = model.predict(X_valid)
    test_pred = model.predict(X_test)
    print("=================" + model_name + "=================")
    train_cm = confusion_matrix(y_train, train_pred)
    valid_cm = confusion_matrix(y_valid, valid_pred)
    test_cm = confusion_matrix(y_test, test_pred)
    i=0
    train_acc_all = np.zeros(6)
    for c in train_cm:
        train_acc_all[i] = c[i]/np.sum(c)
        print("%d train_acc: %.2f" %(i+1, 100*train_acc_all[i]))
        i=i+1
    print("average: %.2f" % (100*np.mean(train_acc_all)))
    i=0
    valid_acc_all = np.zeros(6)
    for c in valid_cm:
        valid_acc_all[i] = c[i]/np.sum(c)
        print("%d valid_acc: %.2f" %(i+1, 100*valid_acc_all[i]))
        i=i+1
    print("average: %.2f" % (100*np.mean(valid_acc_all)))
    i=0
    test_acc_all = np.zeros(6)
    for c in test_cm:
        test_acc_all[i] = c[i]/np.sum(c)
        print("%d test_acc: %.2f" %(i+1, 100*test_acc_all[i]))
        i=i+1
    print("average: %.2f" % (100*np.mean(test_acc_all)))
# - 计算NCE（1-5）

population_best_weight = NCE_weights

classifier_num = 5

# 所有学习器都输出概率向量，最后投票
y_train_pred_proba_all = []
y_valid_pred_proba_all = []
y_test_pred_proba_all = []

# 取训练好的模型，计算各模型”验证集“上输出概率向量
for model in models:
    train_pred_proba = model.predict_proba(X_train)
    valid_pred_proba = model.predict_proba(X_valid)
    test_pred_proba = model.predict_proba(X_test)
    y_train_pred_proba_all.append(train_pred_proba)
    y_valid_pred_proba_all.append(valid_pred_proba)
    y_test_pred_proba_all.append(test_pred_proba)
    
y_train_pred_ensemble_proba = np.zeros((len(y_train), 6)) # 初始化集成器概率向量
y_valid_pred_ensemble_proba = np.zeros((len(y_valid), 6)) # 初始化集成器概率向量
y_test_pred_ensemble_proba = np.zeros((len(y_test), 6)) # 初始化集成器概率向量

# 为每一个基学习器乘上权重
for k in range(classifier_num):
    y_train_pred_ensemble_proba += y_train_pred_proba_all[k] * population_best_weight[k]
    y_valid_pred_ensemble_proba += y_valid_pred_proba_all[k] * population_best_weight[k]
    y_test_pred_ensemble_proba += y_test_pred_proba_all[k] * population_best_weight[k]
y_train_pred_ensemble = np.argmax(y_train_pred_ensemble_proba, axis=1)
y_valid_pred_ensemble = np.argmax(y_valid_pred_ensemble_proba, axis=1)
y_test_pred_ensemble = np.argmax(y_test_pred_ensemble_proba, axis=1)

# 计算各水质等级的得分
print("=================NCE(5)=================")
train_cm = confusion_matrix(y_train, y_train_pred_ensemble)
valid_cm = confusion_matrix(y_valid, y_valid_pred_ensemble)
test_cm = confusion_matrix(y_test, y_test_pred_ensemble)
i=0
train_acc_all = np.zeros(6)
for c in train_cm:
    train_acc_all[i] = c[i]/np.sum(c)
    print("%d train_acc: %.2f" %(i+1, 100*train_acc_all[i]))
    i=i+1
print("average: %.2f" % (100*np.mean(train_acc_all)))
i=0
valid_acc_all = np.zeros(6)
for c in valid_cm:
    valid_acc_all[i] = c[i]/np.sum(c)
    print("%d valid_acc: %.2f" %(i+1, 100*valid_acc_all[i]))
    i=i+1
print("average: %.2f" % (100*np.mean(valid_acc_all)))
i=0
test_acc_all = np.zeros(6)
for c in test_cm:
    test_acc_all[i] = c[i]/np.sum(c)
    print("%d test_acc: %.2f" %(i+1, 100*test_acc_all[i]))
    i=i+1
print("average: %.2f" % (100*np.mean(test_acc_all)))
# - 计算CMAES（1-5）

population_best_weight = CMAES_weights

classifier_num = 5

# 所有学习器都输出概率向量，最后投票
y_train_pred_proba_all = []
y_valid_pred_proba_all = []
y_test_pred_proba_all = []

# 取训练好的模型，计算各模型”验证集“上输出概率向量
for model in models:
    train_pred_proba = model.predict_proba(X_train)
    valid_pred_proba = model.predict_proba(X_valid)
    test_pred_proba = model.predict_proba(X_test)
    y_train_pred_proba_all.append(train_pred_proba)
    y_valid_pred_proba_all.append(valid_pred_proba)
    y_test_pred_proba_all.append(test_pred_proba)
    
y_train_pred_ensemble_proba = np.zeros((len(y_train), 6)) # 初始化集成器概率向量
y_valid_pred_ensemble_proba = np.zeros((len(y_valid), 6)) # 初始化集成器概率向量
y_test_pred_ensemble_proba = np.zeros((len(y_test), 6)) # 初始化集成器概率向量

# 为每一个基学习器乘上权重
for k in range(classifier_num):
    y_train_pred_ensemble_proba += y_train_pred_proba_all[k] * population_best_weight[k]
    y_valid_pred_ensemble_proba += y_valid_pred_proba_all[k] * population_best_weight[k]
    y_test_pred_ensemble_proba += y_test_pred_proba_all[k] * population_best_weight[k]
y_train_pred_ensemble = np.argmax(y_train_pred_ensemble_proba, axis=1)
y_valid_pred_ensemble = np.argmax(y_valid_pred_ensemble_proba, axis=1)
y_test_pred_ensemble = np.argmax(y_test_pred_ensemble_proba, axis=1)

# 计算各水质等级的得分
print("=================CMAES=================")
train_cm = confusion_matrix(y_train, y_train_pred_ensemble)
valid_cm = confusion_matrix(y_valid, y_valid_pred_ensemble)
test_cm = confusion_matrix(y_test, y_test_pred_ensemble)
i=0
train_acc_all = np.zeros(6)
for c in train_cm:
    train_acc_all[i] = c[i]/np.sum(c)
    print("%d train_acc: %.2f" %(i+1, 100*train_acc_all[i]))
    i=i+1
print("average: %.2f" % (100*np.mean(train_acc_all)))
i=0
valid_acc_all = np.zeros(6)
for c in valid_cm:
    valid_acc_all[i] = c[i]/np.sum(c)
    print("%d valid_acc: %.2f" %(i+1, 100*valid_acc_all[i]))
    i=i+1
print("average: %.2f" % (100*np.mean(valid_acc_all)))
i=0
test_acc_all = np.zeros(6)
for c in test_cm:
    test_acc_all[i] = c[i]/np.sum(c)
    print("%d test_acc: %.2f" %(i+1, 100*test_acc_all[i]))
    i=i+1
print("average: %.2f" % (100*np.mean(test_acc_all)))<jupyter_output>=================CMAES=================
1 train_acc: 81.05
2 train_acc: 96.09
3 train_acc: 93.62
4 train_acc: 94.46
5 train_acc: 90.71
6 train_acc: 96.12
average: 92.01
1 valid_acc: 76.64
2 valid_acc: 94.69
3 valid_acc: 92.15
4 valid_acc: 91.96
5 valid_acc: 78.79
6 valid_acc: 94.64
average: 88.15
1 test_acc: 80.74
2 test_acc: 94.76
3 test_acc: 91.02
4 test_acc: 92.50
5 test_acc: 88.22
6 test_acc: 96.50
average: 90.62<jupyter_text>F1<jupyter_code>population_best_weight = NCE_weights


# 所有学习器都输出概率向量，最后投票
y_train_pred_proba_all = []
y_valid_pred_proba_all = []
y_test_pred_proba_all = []

# 取训练好的模型，计算各模型”验证集“上输出概率向量
for model in models:
    model_name = model.__class__.__name__
    train_pred_proba = model.predict_proba(X_train)
    valid_pred_proba = model.predict_proba(X_valid)
    test_pred_proba = model.predict_proba(X_test)
    train_pred = model.predict(X_train)
    valid_pred = model.predict(X_valid)
    test_pred = model.predict(X_test)
    print("=================" + model_name + "=================")
    print(classification_report(y_train, train_pred, digits=4))
    print(classification_report(y_valid, valid_pred, digits=4))
    print(classification_report(y_test, test_pred, digits=4))

    y_train_pred_proba_all.append(train_pred_proba)
    y_valid_pred_proba_all.append(valid_pred_proba)
    y_test_pred_proba_all.append(test_pred_proba)

    
y_train_pred_ensemble_proba = np.zeros((len(y_train), 6)) # 初始化集成器概率向量
y_valid_pred_ensemble_proba = np.zeros((len(y_valid), 6)) # 初始化集成器概率向量
y_test_pred_ensemble_proba = np.zeros((len(y_test), 6)) # 初始化集成器概率向量

# 为每一个基学习器乘上权重
for k in range(classifier_num):
    y_train_pred_ensemble_proba += y_train_pred_proba_all[k] * population_best_weight[k]
    y_valid_pred_ensemble_proba += y_valid_pred_proba_all[k] * population_best_weight[k]
    y_test_pred_ensemble_proba += y_test_pred_proba_all[k] * population_best_weight[k]
y_train_pred_ensemble = np.argmax(y_train_pred_ensemble_proba, axis=1)
y_valid_pred_ensemble = np.argmax(y_valid_pred_ensemble_proba, axis=1)
y_test_pred_ensemble = np.argmax(y_test_pred_ensemble_proba, axis=1)

# 计算各水质等级的得分
print("=================NCE=================")

print(classification_report(y_train, y_train_pred_ensemble, digits=4))
print(classification_report(y_valid, y_valid_pred_ensemble, digits=4))
print(classification_report(y_test, y_test_pred_ensemble, digits=4))
# 所有学习器都输出概率向量，最后投票
population_best_weight = CMAES_weights

y_train_pred_proba_all = []
y_valid_pred_proba_all = []
y_test_pred_proba_all = []

# 取训练好的模型，计算各模型”验证集“上输出概率向量
for model in models:
    model_name = model.__class__.__name__
    train_pred_proba = model.predict_proba(X_train)
    valid_pred_proba = model.predict_proba(X_valid)
    test_pred_proba = model.predict_proba(X_test)
    train_pred = model.predict(X_train)
    valid_pred = model.predict(X_valid)
    test_pred = model.predict(X_test)
    print("=================" + model_name + "=================")
    print(classification_report(y_train, train_pred, digits=4))
    print(classification_report(y_valid, valid_pred, digits=4))
    print(classification_report(y_test, test_pred, digits=4))

    y_train_pred_proba_all.append(train_pred_proba)
    y_valid_pred_proba_all.append(valid_pred_proba)
    y_test_pred_proba_all.append(test_pred_proba)

    
y_train_pred_ensemble_proba = np.zeros((len(y_train), 6)) # 初始化集成器概率向量
y_valid_pred_ensemble_proba = np.zeros((len(y_valid), 6)) # 初始化集成器概率向量
y_test_pred_ensemble_proba = np.zeros((len(y_test), 6)) # 初始化集成器概率向量

# 为每一个基学习器乘上权重
for k in range(classifier_num):
    y_train_pred_ensemble_proba += y_train_pred_proba_all[k] * population_best_weight[k]
    y_valid_pred_ensemble_proba += y_valid_pred_proba_all[k] * population_best_weight[k]
    y_test_pred_ensemble_proba += y_test_pred_proba_all[k] * population_best_weight[k]
y_train_pred_ensemble = np.argmax(y_train_pred_ensemble_proba, axis=1)
y_valid_pred_ensemble = np.argmax(y_valid_pred_ensemble_proba, axis=1)
y_test_pred_ensemble = np.argmax(y_test_pred_ensemble_proba, axis=1)

# 计算各水质等级的得分
print("=================CMAES=================")

print(classification_report(y_train, y_train_pred_ensemble, digits=4))
print(classification_report(y_valid, y_valid_pred_ensemble, digits=4))
print(classification_report(y_test, y_test_pred_ensemble, digits=4))<jupyter_output>=================LogisticRegression=================
             precision    recall  f1-score   support

          0     0.8654    0.0308    0.0594      1462
          1     0.6703    0.9921    0.8000      7962
          2     0.6432    0.5722    0.6056      5278
          3     0.6217    0.4505    0.5224      3283
          4     0.0000    0.0000    0.0000       893
          5     0.7904    0.7702    0.7802      1288

avg / total     0.6474    0.6662    0.6136     20166

             precision    recall  f1-score   support

          0     0.9375    0.0307    0.0595       488
          1     0.6764    0.9913    0.8042      2655
          2     0.6439    0.5850    0.6130      1759
          3     0.6338    0.4521    0.5277      1095
          4     0.0000    0.0000    0.0000       297
          5     0.7735    0.7879    0.7806       429

avg / total     0.6562    0.6707    0.6181      6723

             precision    recall  f1-score   support

          0     0.9333    0.0574    0.1[...]<jupyter_text>人为设定了10个参数的初值x0=10*[0]，sigma0=0.5, 算法包中对sigma0的描述为：http://cma.gforge.inria.fr/apidocs-pycma/cma.evolution_strategy.CMAEvolutionStrategy.html> initial standard deviation. The problem variables should have been scaled, such that a single standard deviation on all variables is useful and the optimum is expected to lie within about x0 +- 3*sigma0. See also options scaling_of_variables. Often one wants to check for solutions close to the initial point. This allows, for example, for an easier check of consistency of the objective function and its interfacing with the optimizer. In this case, a much smaller sigma0 is advisable.CMAES 知乎算法专栏https://zhuanlan.zhihu.com/p/31193293<jupyter_code># 算法默认参数

cma.CMAOptions.defaults()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13858534131955558, 'fraction_numerical': 0.0374011247771021, 'mean_word_length': 4.510707986898463, 'pattern_counts': {'":': 16, '<': 41, '<?xml version=': 0, '>': 42, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '306246', 'license': 'MIT', 'n_tokens_mistral': 9613, 'n_tokens_neox': 9449, 'n_words': 1272, 'path': 'notebook/NCE_5.ipynb', 'repo_name': 'chenkangyang/ES-Ensemble', 'size': '21871'}