<jupyter_start><jupyter_text>Инициализация<jupyter_code>#@markdown - **Монтирование GoogleDrive** 
from google.colab import drive
drive.mount('GoogleDrive')
# #@markdown - **Размонтирование**
# !fusermount -u GoogleDrive<jupyter_output><empty_output><jupyter_text>Область кодов<jupyter_code>#@title Expectation Maximization algorithm { display-mode: "both" }
# В программе реализована оценка параметров распределения Бернулли с помощью ЕМ-алгоритма
# Данные здесь 1D. Для примера многомерных данных, пожалуйста, обратитесь к программе модели гауссовых смесей
#@markdown [Оригинальная программа](https://github.com/wzyonggege/statistical-learning-method/blob/master/EM/em.ipynb)
# coding: utf-8
import numpy as np<jupyter_output><empty_output><jupyter_text>Функция максимального правдоподобия (Maximum likehood function):> $P(y|\pi,\theta)=\sum_{k=1}^{K}\pi_{k}\prod_{d=1}^{D}\theta_{d}^{y_{d}}(1-\theta_{d})^{1-y_{d}}$> если $K=2$ ，определяем $\pi=[\pi, 1-\pi]$, $\theta=[p, q]$, $Y=[y_{1},...,y_{N}]$ :> $P(Y|\pi,\theta)=\sum_{n=1}^{N}\left [\prod_{d=1}^{D}\pi p^{y_{d}}(1-p)^{1-y_{d}}+\prod_{d=1}^{D}(1-\pi) q^{y_{d}}(1-q)^{1-y_{d}}  \right ]$ особенно，когда $D=1$ ， E-step:> $\mu_{i+1}=\frac{\pi (p_{i})^{y_{i}}(1-(p_{i}))^{1-y_{i}}}{\pi (p_{i})^{y_{i}}(1-(p_{i}))^{1-y_{i}}+(1-\pi) (q_{i})^{y_{i}}(1-(q_{i}))^{1-y_{i}}} $ M-step:> $\pi_{i+1} =\frac{1}{N} \sum_{j=1}^{N}\mu_{i+1,j}$> $p_{i+1}=\frac{\sum_{j=1}^{N}\mu_{i+1,j} (y_{j})}{\sum_{j=1}^{N}\mu_{i+1,j}}$> $q_{i+1}=\frac{\sum_{j=1}^{N}(1-\mu_{i+1,j}(y_{j}))}{\sum_{j=1}^{N}(1-\mu_{i+1,j})}$<jupyter_code>#@markdown - **EM класс**
class EM:
    def __init__(self, prob):
        self.pro_A, self.pro_B, self.pro_C = prob
        
    #@markdown - **e_step**
    def pro_1(self, x):
        return self.pro_A * np.power(self.pro_B, x) * np.power((1 - self.pro_B), 1 - x)
    def pro_2(self, x):
        return (1 - self.pro_A) * np.power(self.pro_C, x) * np.power((1 - self.pro_C), 1 - x)
    def pmf(self, x):
        return self.pro_1(x) / (self.pro_1(x) + self.pro_2(x))
    
    #@markdown - **m_step**
    def fit(self, data, max_error=1e-5):
        count = data.shape[0]
        print('init prob:{}, {}, {}'.format(self.pro_A, self.pro_B, self.pro_C))
        d = 0
        while True:
            d += 1
            PMF = [self.pmf(x) for x in data]
            pro_A = 1/ count * sum(PMF)
            pro_B = sum([PMF[k] * data[k] for k in range(count)]) / sum([PMF[k] for k in range(count)])
            pro_C = sum([(1 - PMF[k]) * data[k] for k in range(count)]) / sum([(1 - PMF[k]) for k in range(count)])
            error = abs(pro_A - self.pro_A) + abs(pro_B - self.pro_B) + abs(pro_C - self.pro_C)
            print_list = [d, pro_A, pro_B, pro_C, error]
            print('Step {0[0]},  pro_a:{0[1]:.3f}, pro_b:{0[2]:.3f}, pro_c:{0[3]:.3f}, error: {0[4]:.6f}.'.format(print_list))
            self.pro_A = pro_A
            self.pro_B = pro_B
            self.pro_C = pro_C
            if error < max_error: break
    def mlf(self, y):
        return self.pro_1(y) + self.pro_2(y)
        
#@markdown - **Генерация одномерных данных, которые принадлежат к распределению Бернулли**
data = np.random.binomial(1, 0.2, size=[200, ])
em = EM(prob=[0.2, 0.3, 0.4])
em.fit(data)
data
em.mlf(0)
em.mlf(1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.23144501666161768, 'fraction_numerical': 0.02726446531354135, 'mean_word_length': 3.592489568845619, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 20, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '187296', 'license': 'MIT', 'n_tokens_mistral': 1491, 'n_tokens_neox': 1513, 'n_words': 240, 'path': 'notebooks(colab)/Classic_models/EM_RU.ipynb', 'repo_name': 'jswanglp/MyML', 'size': '3301'}