<jupyter_start><jupyter_text>Matrix FactorizationMatrix factorization is one of model-based algorithm (or Latent Factor Model). Using the nature of the matrix.  Define- Basic concept of matrix factorization is `matrix completion`.- Fill the below image's `?` is main purpose of matrix completion.- Matrix's row, column is factor of concept (e.g. Row: user, Column: Movie, Value: Rating)- And matrix factorization divide it two parts(P, Q). And they called `Latent Factor`.    - $ R \approx P X Q^T = \hat{R} $    - And Latent factor has their own dimension `k`.    - $ \hat{r_{ij}} = p_i^T q_j = \sum_{k=1}^k p_{ik}q_{kj} $    - The example of using latent factor        - **e.g.** When we have (User, Movie, Rating) matrix, we implicitically knows about every user has prefered genre. In this situation, user latent factor's dimension(k) maybe works like `prefer genre`. if k=3, each dimension value represent preference score of each genre. (romance = -1, action = 2, comedy = 10) ---- Cost function & Train- Cost function of Matrix Factorization is below.$$ Cost = \sum_{i,u \in R_{train}} (r(i,u) - \hat{r}(i,u) )^2 + \lambda(b(i)^2 + b(u)^2 + ||p(i)||^2 ||q(u)||^2) $$- And we can use Gradient Descent.$$ \grave{p_{ik}} = p_{ik} +\alpha \frac{dCost}{dp}$$$$ \grave{q_{kj}} = q_{kj} +\alpha \frac{dCost}{dq}$$- Here is my process of deviation<jupyter_code>import numpy as np

# 7X5 Matrix
R = np.array([
        [1, 0, 0, 1, 3],
        [2, 0, 3, 1, 1],
        [1, 2, 0, 5, 0],
        [1, 0, 0, 4, 4],
        [2, 1, 5, 4, 0],
        [5, 1, 5, 4, 0],
        [0, 0, 0, 1, 0],
    ])

# P, Q is (7 X k), (k X 5) matrix
P = np.random.normal(size=(R.shape[0], 4))
Q = np.random.normal(size=(R.shape[1], 4))

# b_P, b_Q is (7 X 1), (5 X 1)
b_P = np.ones(R.shape[0])
b_Q = np.ones(R.shape[1])
b = np.mean(R[np.where(R != 0)])
P
Q
R_hat = b + b_P[:, np.newaxis] + b_Q[np.newaxis:, ] + P.dot(Q.T)
R_hat<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17908902691511386, 'fraction_numerical': 0.03157349896480331, 'mean_word_length': 3.2955555555555556, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '525763', 'license': 'MIT', 'n_tokens_mistral': 782, 'n_tokens_neox': 720, 'n_words': 259, 'path': '03-linear-algebra/03-matrix-factorization.ipynb', 'repo_name': 'yoonkt200/ml-theory-python', 'size': '1932'}