<commit_before><!DOCTYPE HTML>
<html>
    <head>
        <title>General Linear Models</title>

        <script src="js/jquery.min.js"></script>
        <script src="js/jquery.scrolly.min.js"></script>
        <script src="js/jquery.scrollgress.min.js"></script>
        <script src="js/skel.min.js"></script>
        <script src="js/skel-layers.min.js"></script>
        <script src="js/init.js"></script>
        <script src='https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'></script>

        <noscript>
            <link rel="stylesheet" href="css/skel.css" />
            <link rel="stylesheet" href="css/style.css" />
            <link rel="stylesheet" href="css/style-wide.css" />
            <link rel="stylesheet" href="css/style-noscript.css" />
        </noscript>

        <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
        <script src="js/smoothscroll.js"></script>

        <link rel="stylesheet" type="text/css" media="screen" href="http://cdnjs.cloudflare.com/ajax/libs/fancybox/1.3.4/jquery.fancybox-1.3.4.css" />
        <style type="text/css">
            a.fancybox img {
                border: none;
            }
        </style>

        <script>
            (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
            (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
            m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
            })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

            ga('create', 'UA-68428630-1', 'auto');
            ga('send', 'pageview');

        </script>

    </head>


    <body class="index">

       <!-- Header -->
        <header id="header" style="position: fixed; top: 0px; left: 0px; right: 0px; z-index: 1030;">
            <h1 id="logo"><a href="index.html">Home</a></h1>
            <nav id="nav">
                <ul>
                    <li><a href="#top" class="smoothScroll">^</a></li>
                    <li><a href="#contact" class="smoothScroll">Contact</a></li>
                </ul>
            </nav>
        </header>

        <!-- Main -->
        <article id="main">

            <!-- Emoji -->
            <div id="top">
                <section class="wrapper style3 container">

                    <header>
                        <h2><center>General Linear Models</center></h2>
                    </header>

                    <!-- Text -->
                    <h3>Simple Linear Regression</h3>
                    <p>
                        A line can be described by the equation \(y = mx + b\),
                        where \(b\) is the \(y\)-intercept and \(m\) is the
                        slope. Simple linear regression uses this idea to
                        describe the <em>relationship</em> between two
                        variables. The process uses the data to find the line
                        that "best" fits it.
                    </p>
                    <p>
                        Because each observation has its own \(x\) and \(y\)
                        value, we use subscripts to denote them. In addition,
                        unless the data form a single straight line, we will
                        need to introduce an additional concept&mdash;errors.
                    </p>
                    <p>
                        Relative to the <em>true</em> line that describes \(x\)
                        and \(y\), the observed data will be distributed both
                        above and below it. The differences between these points
                        and the line is referred to as the "error." To capture
                        this, we add a new term to our equation.

                        \[y_i = \beta_0 + \beta_1 x_i + \epsilon_i\]

                        In this form, in which the terms have been slightly
                        reordered from the equation above, \(\beta_0\)
                        represents the \(y\)-intercept, \(\beta_1\) represents
                        the slope, and \(\epsilon_i\) represents the error.
                        Notice that the \(y\), \(x\), and \(\epsilon\) terms all
                        have subscripts, but the \(\beta\)s do not. This implies
                        that there is a <em>single</em> \(y\)-intercept and
                        slope&mdash;the true line.
                    </p>
                    <p>
                        While we have specified the relationship we believe the
                        data have, the model is not <em>fully specified</em>.
                        For this, we must be explicit about our assumptions of
                        \(\epsilon\). We can, for example, assume that
                        \(\epsilon\) are independent and identically distributed
                        with a zero mean and \(\sigma^2\) variance.
                    </p>
                    <p>
                        Together, the equation and the assumptions about
                        \(\epsilon\) describe the model.
                    </p>
                    <h3>Matrix Form</h3>
                    <p>
                        It is useful to specify our model in matrix form. We can
                        think of \(y_i\), \(x_i\), and \(\epsilon_i\) as being
                        vectors with \(n\) values, where \(i \in 1,\ldots,n\).
                        \(y_i\), for example, can be written as \(\vec{y}\). Our
                        equation becomes:
                    </p>
                    <p>
                        \[\vec{y} = \beta_0 + \beta_1 \vec{x} + \vec{\epsilon}\]
                    </p>
                    <p>
                        Using this notation, the \(\beta\) values are scalars.
                        However, with what we've described thus far, \(\beta_0\)
                        doesn't have a vector of values to be multiplied with.
                        So, we add a vector of ones of length \(n\).
                    </p>
                    <p>
                        Our <em>system</em> of \(n\) equations can be
                        represented as the following matrix addition equation.
                    </p>
                    <p>
                        \begin{split}
                            \left[\begin{matrix}y_1\\ \vdots\\ y_n\end{matrix}\right] =
                            \beta_0 \left[\begin{matrix}1\\ \vdots\\ 1\end{matrix}\right] +
                            \beta_1 \left[\begin{matrix}x_1\\ \vdots\\ x_n\end{matrix}\right] +
                            \left[\begin{matrix}\epsilon_1\\ \vdots\\ \epsilon_n\end{matrix}\right]
                        \end{split}
                    </p>
                    <p>
                        We can restate this in terms of matrix multiplication.
                    </p>
                    <p>
                        \begin{split}
                            \left[\begin{matrix}y_1\\ \vdots\\ y_n\end{matrix}\right] =
                            \left[\begin{matrix}1 & x_1\\ \vdots & \vdots\\ 1 & x_n\end{matrix}\right]
                            \left[\begin{matrix}\beta_0\\ \beta_1\end{matrix}\right] +
                            \left[\begin{matrix}\epsilon_1\\ \vdots\\ \epsilon_n\end{matrix}\right]
                        \end{split}
                    </p>
                    <p>
                        Here, we've simply combined the vector of ones and
                        \(x_i\) into a matrix, which we refer to as \(X\), and
                        combined the \(\beta\)s into a vector, which we'll call
                        \(\vec{\beta}\). In this example, \(X\) is an \(m\) by
                        \(p\) matrix, where \(p = 2\), and \(\vec{\beta}\)
                        is a vector of length \(2\). Thus, we're able to
                        multiply both terms.
                    </p>
                    <p>
                        This can be rewritten as follows.
                    </p>
                    <p>
                        \[\vec{y} = X\vec{\beta} + \vec{\epsilon} \]
                    </p>
                    <p>
                        What makes this formulation powerful is that we can add
                        additional \(x\) variables and associated \(\beta\)
                        coefficients.
                    </p>

                </section>
            </div>
        </article>

        <!-- Footer -->
        <div id="contact">
            <footer id="footer">

                <h2><strong>Contact</strong></h2>
                <ul class="icons">
                    <li><a href="http://www.linkedin.com/in/juanshishido/" class="icon circle fa-linkedin" target="_blank"><span class="label">LinkedIn</span></a></li>
                    <li><a href="https://twitter.com/juanshishido" class="icon circle fa-twitter" target="_blank"><span class="label">Twitter</span></a></li>
                    <li><a href="https://github.com/juanshishido" class="icon circle fa-github" target="_blank"><span class="label">Github</span></a></li>
                </ul><commit_msg>Added design matrix and parameter vector, multiple regression discussion<commit_after><!DOCTYPE HTML>
<html>
    <head>
        <title>General Linear Models</title>

        <script src="js/jquery.min.js"></script>
        <script src="js/jquery.scrolly.min.js"></script>
        <script src="js/jquery.scrollgress.min.js"></script>
        <script src="js/skel.min.js"></script>
        <script src="js/skel-layers.min.js"></script>
        <script src="js/init.js"></script>
        <script src='https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'></script>

        <noscript>
            <link rel="stylesheet" href="css/skel.css" />
            <link rel="stylesheet" href="css/style.css" />
            <link rel="stylesheet" href="css/style-wide.css" />
            <link rel="stylesheet" href="css/style-noscript.css" />
        </noscript>

        <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
        <script src="js/smoothscroll.js"></script>

        <link rel="stylesheet" type="text/css" media="screen" href="http://cdnjs.cloudflare.com/ajax/libs/fancybox/1.3.4/jquery.fancybox-1.3.4.css" />
        <style type="text/css">
            a.fancybox img {
                border: none;
            }
        </style>

        <script>
            (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
            (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
            m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
            })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

            ga('create', 'UA-68428630-1', 'auto');
            ga('send', 'pageview');

        </script>

    </head>


    <body class="index">

       <!-- Header -->
        <header id="header" style="position: fixed; top: 0px; left: 0px; right: 0px; z-index: 1030;">
            <h1 id="logo"><a href="index.html">Home</a></h1>
            <nav id="nav">
                <ul>
                    <li><a href="#top" class="smoothScroll">^</a></li>
                    <li><a href="#contact" class="smoothScroll">Contact</a></li>
                </ul>
            </nav>
        </header>

        <!-- Main -->
        <article id="main">

            <!-- Emoji -->
            <div id="top">
                <section class="wrapper style3 container">

                    <header>
                        <h2><center>General Linear Models</center></h2>
                    </header>

                    <!-- Text -->
                    <h3>Simple Linear Regression</h3>
                    <p>
                        A line can be described by the equation \(y = mx + b\),
                        where \(b\) is the \(y\)-intercept and \(m\) is the
                        slope. Simple linear regression uses this idea to
                        describe the <em>relationship</em> between two
                        variables&mdash;one dependent, the \(y\), and one
                        explanatory, the \(x\). The process uses the data to
                        find the line that "best" fits it.
                    </p>
                    <p>
                        Because each observation has its own \(x\) and \(y\)
                        value, we use subscripts to denote them. In addition,
                        unless the data form a single straight line, we will
                        need to introduce an additional concept&mdash;errors.
                    </p>
                    <p>
                        Relative to the <em>true</em> line that describes \(x\)
                        and \(y\), the observed data will be distributed both
                        above and below it. The differences between these points
                        and the line is referred to as the "error." To capture
                        this, we add a new term to our equation.

                        \[y_i = \beta_0 + \beta_1 x_i + \epsilon_i\]

                        In this form, in which the terms have been slightly
                        reordered from the equation above, \(\beta_0\)
                        represents the \(y\)-intercept, \(\beta_1\) represents
                        the slope, and \(\epsilon_i\) represents the error.
                        Notice that the \(y\), \(x\), and \(\epsilon\) terms all
                        have subscripts, but the \(\beta\)s do not. This implies
                        that there is a <em>single</em> \(y\)-intercept and
                        slope&mdash;the true line.
                    </p>
                    <p>
                        While we have specified the relationship we believe the
                        data have, the model is not <em>fully specified</em>.
                        For this, we must be explicit about our assumptions of
                        \(\epsilon\). We can, for example, assume that
                        \(\epsilon\) are independent and identically distributed
                        with a zero mean and \(\sigma^2\) variance.
                    </p>
                    <p>
                        Together, the equation and the assumptions about
                        \(\epsilon\) describe the model.
                    </p>
                    <h3>Matrix Form</h3>
                    <p>
                        It is useful to specify our model in matrix form. We can
                        think of \(y_i\), \(x_i\), and \(\epsilon_i\) as being
                        vectors with \(n\) values, where \(i \in 1,\ldots,n\).
                        \(y_i\), for example, can be written as \(\vec{y}\). Our
                        equation becomes:
                    </p>
                    <p>
                        \[\vec{y} = \beta_0 + \beta_1 \vec{x} + \vec{\epsilon}\]
                    </p>
                    <p>
                        Using this notation, the \(\beta\) values are scalars.
                        However, with what we've described thus far, \(\beta_0\)
                        doesn't have a vector of values to be multiplied with.
                        So, we add a vector of ones of length \(n\).
                    </p>
                    <p>
                        Our <em>system</em> of \(n\) equations can be
                        represented as the following matrix addition equation.
                    </p>
                    <p>
                        \begin{split}
                            \left[\begin{matrix}y_1\\ \vdots\\ y_n\end{matrix}\right] =
                            \beta_0 \left[\begin{matrix}1\\ \vdots\\ 1\end{matrix}\right] +
                            \beta_1 \left[\begin{matrix}x_1\\ \vdots\\ x_n\end{matrix}\right] +
                            \left[\begin{matrix}\epsilon_1\\ \vdots\\ \epsilon_n\end{matrix}\right]
                        \end{split}
                    </p>
                    <p>
                        We can restate this in terms of matrix multiplication.
                    </p>
                    <p>
                        \begin{split}
                            \left[\begin{matrix}y_1\\ \vdots\\ y_n\end{matrix}\right] =
                            \left[\begin{matrix}1 & x_1\\ \vdots & \vdots\\ 1 & x_n\end{matrix}\right]
                            \left[\begin{matrix}\beta_0\\ \beta_1\end{matrix}\right] +
                            \left[\begin{matrix}\epsilon_1\\ \vdots\\ \epsilon_n\end{matrix}\right]
                        \end{split}
                    </p>
                    <p>
                        Here, we've simply combined the vector of ones and
                        \(x_i\) into a matrix, which we refer to as \(X\),
                        called the <em>design matrix</em>, and
                        combined the \(\beta\)s into a column vector, which
                        we refer to as \(\vec{\beta}\), called the <em>parameter
                        vector</em>. In this example, \(X\) is an
                        \(n \times p\) matrix and \(\vec{\beta}\) can be thought
                        of as a \(p \times 1\) matrix. Because the number of
                        columns in \(X\) equals the number of rows in
                        \(\vec{\beta}\), they can be multiplied.
                    </p>
                    <p>
                        This can be rewritten as follows.
                    </p>
                    <p>
                        \[\vec{y} = X\vec{\beta} + \vec{\epsilon} \]
                    </p>
                    <h3>Multiple Regression</h3>
                    <p>
                        Often, outcomes for our variable of interest, \(y_i\),
                        depend on two or more variables. This is referred to as
                        a multiple linear regression model, which, for the case
                        of two variables, can be written as:
                    </p>
                    <p>
                        \[y_i = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + \epsilon_i\]
                    </p>
                    <p>
                        This model, as any model with an arbitrary number of
                        parameters, can still be expressed as:
                    </p>
                    <p>
                        \[\vec{y} = X\vec{\beta} + \vec{\epsilon} \]
                    </p>
                        For each parameter, \(p\), add a column to \(X\) and an
                        element to \(\vec{\beta}\).
                    </p>

                </section>
            </div>
        </article>

        <!-- Footer -->
        <div id="contact">
            <footer id="footer">

                <h2><strong>Contact</strong></h2>
                <ul class="icons">
                    <li><a href="http://www.linkedin.com/in/juanshishido/" class="icon circle fa-linkedin" target="_blank"><span class="label">LinkedIn</span></a></li>
                    <li><a href="https://twitter.com/juanshishido" class="icon circle fa-twitter" target="_blank"><span class="label">Twitter</span></a></li>
                    <li><a href="https://github.com/juanshishido" class="icon circle fa-github" target="_blank"><span class="label">Github</span></a></li>
                </ul>
{'fraction_non_alphanumeric': 0.13322788039595876, 'fraction_numerical': 0.006837432391060313, 'mean_word_length': 1.153972964061985, 'pattern_counts': {'":': 0, '<': 302, '<?xml version=': 0, '>': 302, 'https://': 6, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6280288', 'n_tokens_mistral': 5373, 'n_tokens_neox': 4947, 'n_words': 1395}