<commit_before>Linear Algebra (`scipy.linalg`)
===============================

.. sectionauthor:: <NAME>

.. currentmodule: scipy

When SciPy is built using the optimized ATLAS LAPACK and BLAS
libraries, it has very fast linear algebra capabilities. If you dig
deep enough, all of the raw lapack and blas libraries are available
for your use for even more speed. In this section, some easier-to-use
interfaces to these routines are described.

All of these linear algebra routines expect an object that can be
converted into a 2-dimensional array. The output of these routines is
also a two-dimensional array. There is a matrix class defined in
Numpy, which you can initialize with an appropriate Numpy array in
order to get objects for which multiplication is matrix-multiplication
instead of the default, element-by-element multiplication.


Matrix Class
------------

The matrix class is initialized with the SciPy command :obj:`mat`
which is just convenient short-hand for :class:`matrix
<numpy.matrix>`. If you are going to be doing a lot of matrix-math, it
is convenient to convert arrays into matrices using this command. One
advantage of using the :func:`mat` command is that you can enter
two-dimensional matrices using MATLAB-like syntax with commas or
spaces separating columns and semicolons separting rows as long as the
matrix is placed in a string passed to :obj:`mat` .


Basic routines
--------------


Finding Inverse
^^^^^^^^^^^^^^^

The inverse of a matrix :math:`\mathbf{A}` is the matrix
:math:`\mathbf{B}` such that :math:`\mathbf{AB}=\mathbf{I}` where
:math:`\mathbf{I}` is the identity matrix consisting of ones down the
main diagonal.  Usually :math:`\mathbf{B}` is denoted
:math:`\mathbf{B}=\mathbf{A}^{-1}` . In SciPy, the matrix inverse of
the Numpy array, A, is obtained using :obj:`linalg.inv` ``(A)`` , or
using ``A.I`` if ``A`` is a Matrix. For example, let

.. math::
   :nowrap:

    \[ \mathbf{A=}\left[\begin{array}{ccc} 1 & 3 & 5\\ 2 & 5 & 1\\ 2 & 3 & 8\end{array}\right]\]

then

.. math::
   :nowrap:

    \[ \mathbf{A^{-1}=\frac{1}{25}\left[\begin{array}{ccc} -37 & 9 & 22\\ 14 & 2 & -9\\ 4 & -3 & 1\end{array}\right]=\left[\begin{array}{ccc} -1.48 & 0.36 & 0.88\\ 0.56 & 0.08 & -0.36\\ 0.16 & -0.12 & 0.04\end{array}\right].}\]

The following example demonstrates this computation in SciPy

    >>> A = mat('[1 3 5; 2 5 1; 2 3 8]')
    >>> A
    matrix([[1, 3, 5],
            [2, 5, 1],
            [2, 3, 8]])
    >>> A.I
    matrix([[-1.48,  0.36,  0.88],
            [ 0.56,  0.08, -0.36],
            [ 0.16, -0.12,  0.04]])
    >>> from scipy import linalg
    >>> linalg.inv(A)
    array([[-1.48,  0.36,  0.88],
           [ 0.56,  0.08, -0.36],
           [ 0.16, -0.12,  0.04]])
    
Solving linear system
^^^^^^^^^^^^^^^^^^^^^

Solving linear systems of equations is straightforward using the scipy
command :obj:`linalg.solve`. This command expects an input matrix and
a right-hand-side vector. The solution vector is then computed. An
option for entering a symmetrix matrix is offered which can speed up
the processing when applicable.  As an example, suppose it is desired
to solve the following simultaneous equations:

.. math::
   :nowrap:

    \begin{eqnarray*} x+3y+5z & = & 10\\ 2x+5y+z & = & 8\\ 2x+3y+8z & = & 3\end{eqnarray*}

We could find the solution vector using a matrix inverse:

.. math::
   :nowrap:

    \[ \left[\begin{array}{c} x\\ y\\ z\end{array}\right]=\left[\begin{array}{ccc} 1 & 3 & 5\\ 2 & 5 & 1\\ 2 & 3 & 8\end{array}\right]^{-1}\left[\begin{array}{c} 10\\ 8\\ 3\end{array}\right]=\frac{1}{25}\left[\begin{array}{c} -232\\ 129\\ 19\end{array}\right]=\left[\begin{array}{c} -9.28\\ 5.16\\ 0.76\end{array}\right].\]

However, it is better to use the linalg.solve command which can be
faster and more numerically stable. In this case it however gives the
same answer as shown in the following example:

    >>> A = mat('[1 3 5; 2 5 1; 2 3 8]')
    >>> b = mat('[10;8;3]')
    >>> A.I*b
    matrix([[-9.28],
            [ 5.16],
            [ 0.76]])
    >>> linalg.solve(A,b)
    array([[-9.28],
           [ 5.16],
           [ 0.76]])


Finding Determinant
^^^^^^^^^^^^^^^^^^^

The determinant of a square matrix :math:`\mathbf{A}` is often denoted
:math:`\left|\mathbf{A}\right|` and is a quantity often used in linear
algebra. Suppose :math:`a_{ij}` are the elements of the matrix
:math:`\mathbf{A}` and let :math:`M_{ij}=\left|\mathbf{A}_{ij}\right|`
be the determinant of the matrix left by removing the
:math:`i^{\textrm{th}}` row and :math:`j^{\textrm{th}}` column from
:math:`\mathbf{A}` . Then for any row :math:`i,`

.. math::
   :nowrap:

    \[ \left|\mathbf{A}\right|=\sum_{j}\left(-1\right)^{i+j}a_{ij}M_{ij}.\]

This is a recursive way to define the determinant where the base case
is defined by accepting that the determinant of a :math:`1\times1` matrix is the only matrix element. In SciPy the determinant can be
calculated with :obj:`linalg.det` . For example, the determinant of

.. math::
   :nowrap:

    \[ \mathbf{A=}\left[\begin{array}{ccc} 1 & 3 & 5\\ 2 & 5 & 1\\ 2 & 3 & 8\end{array}\right]\]

is

.. math::
   :nowrap:

    \begin{eqnarray*} \left|\mathbf{A}\right| & = & 1\left|\begin{array}{cc} 5 & 1\\ 3 & 8\end{array}\right|-3\left|\begin{array}{cc} 2 & 1\\ 2 & 8\end{array}\right|+5\left|\begin{array}{cc} 2 & 5\\ 2 & 3\end{array}\right|\\  & = & 1\left(5\cdot8-3\cdot1\right)-3\left(2\cdot8-2\cdot1\right)+5\left(2\cdot3-2\cdot5\right)=-25.\end{eqnarray*}

In SciPy this is computed as shown in this example:

    >>> A = mat('[1 3 5; 2 5 1; 2 3 8]')
    >>> linalg.det(A)
    -25.000000000000004


Computing norms
^^^^^^^^^^^^^^^

Matrix and vector norms can also be computed with SciPy. A wide range
of norm definitions are available using different parameters to the
order argument of :obj:`linalg.norm` . This function takes a rank-1
(vectors) or a rank-2 (matrices) array and an optional order argument
(default is 2). Based on these inputs a vector or matrix norm of the
requested order is computed.

For vector *x* , the order parameter can be any real number including
``inf`` or ``-inf``. The computed norm is

.. math::
   :nowrap:

    \[ \left\Vert \mathbf{x}\right\Vert =\left\{ \begin{array}{cc} \max\left|x_{i}\right| & \textrm{ord}=\textrm{inf}\\ \min\left|x_{i}\right| & \textrm{ord}=-\textrm{inf}\\ \left(\sum_{i}\left|x_{i}\right|^{\textrm{ord}}\right)^{1/\textrm{ord}} & \left|\textrm{ord}\right|<\infty.\end{array}\right.\]



For matrix :math:`\mathbf{A}` the only valid values for norm are :math:`\pm2,\pm1,` :math:`\pm` inf, and 'fro' (or 'f') Thus,

.. math::
   :nowrap:

    \[ \left\Vert \mathbf{A}\right\Vert =\left\{ \begin{array}{cc} \max_{i}\sum_{j}\left|a_{ij}\right| & \textrm{ord}=\textrm{inf}\\ \min_{i}\sum_{j}\left|a_{ij}\right| & \textrm{ord}=-\textrm{inf}\\ \max_{j}\sum_{i}\left|a_{ij}\right| & \textrm{ord}=1\\ \min_{j}\sum_{i}\left|a_{ij}\right| & \textrm{ord}=-1\\ \max\sigma_{i} & \textrm{ord}=2\\ \min\sigma_{i} & \textrm{ord}=-2\\ \sqrt{\textrm{trace}\left(\mathbf{A}^{H}\mathbf{A}\right)} & \textrm{ord}=\textrm{'fro'}\end{array}\right.\]

where :math:`\sigma_{i}` are the singular values of :math:`\mathbf{A}` .

Examples:

    >>> A=mat('[1, 2; 3, 4]')
    >>> A
    matrix([[1, 2],
            [3, 4]])
    >>> linalg.norm(A)
    5.4772255750516612
    >>> linalg.norm(A,'fro') #'fro' is default
    5.4772255750516612
    >>> linalg.norm(A,1)
    6
    >>> linalg.norm(A,-1)
    4
    >>> linalg.norm(A,inf)
    7


Solving linear least-squares problems and pseudo-inverses
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Linear least-squares problems occur in many branches of applied
mathematics. In this problem a set of linear scaling coefficients is
sought that allow a model to fit data. In particular it is assumed
that data :math:`y_{i}` is related to data :math:`\mathbf{x}_{i}`
through a set of coefficients :math:`c_{j}` and model functions
:math:`f_{j}\left(\mathbf{x}_{i}\right)` via the model

.. math::<commit_msg>better comments to example, use np.array instead of matrix<commit_after>Linear Algebra (`scipy.linalg`)
===============================

.. sectionauthor:: <NAME>

.. currentmodule: scipy

When SciPy is built using the optimized ATLAS LAPACK and BLAS
libraries, it has very fast linear algebra capabilities. If you dig
deep enough, all of the raw lapack and blas libraries are available
for your use for even more speed. In this section, some easier-to-use
interfaces to these routines are described.

All of these linear algebra routines expect an object that can be
converted into a 2-dimensional array. The output of these routines is
also a two-dimensional array. There is a matrix class defined in
Numpy, which you can initialize with an appropriate Numpy array in
order to get objects for which multiplication is matrix-multiplication
instead of the default, element-by-element multiplication.


Matrix Class
------------

The matrix class is initialized with the SciPy command :obj:`mat`
which is just convenient short-hand for :class:`matrix
<numpy.matrix>`. If you are going to be doing a lot of matrix-math, it
is convenient to convert arrays into matrices using this command. One
advantage of using the :func:`mat` command is that you can enter
two-dimensional matrices using MATLAB-like syntax with commas or
spaces separating columns and semicolons separting rows as long as the
matrix is placed in a string passed to :obj:`mat` .


Basic routines
--------------


Finding Inverse
^^^^^^^^^^^^^^^

The inverse of a matrix :math:`\mathbf{A}` is the matrix
:math:`\mathbf{B}` such that :math:`\mathbf{AB}=\mathbf{I}` where
:math:`\mathbf{I}` is the identity matrix consisting of ones down the
main diagonal.  Usually :math:`\mathbf{B}` is denoted
:math:`\mathbf{B}=\mathbf{A}^{-1}` . In SciPy, the matrix inverse of
the Numpy array, A, is obtained using :obj:`linalg.inv` ``(A)`` , or
using ``A.I`` if ``A`` is a Matrix. For example, let

.. math::
   :nowrap:

    \[ \mathbf{A=}\left[\begin{array}{ccc} 1 & 3 & 5\\ 2 & 5 & 1\\ 2 & 3 & 8\end{array}\right]\]

then

.. math::
   :nowrap:

    \[ \mathbf{A^{-1}=\frac{1}{25}\left[\begin{array}{ccc} -37 & 9 & 22\\ 14 & 2 & -9\\ 4 & -3 & 1\end{array}\right]=\left[\begin{array}{ccc} -1.48 & 0.36 & 0.88\\ 0.56 & 0.08 & -0.36\\ 0.16 & -0.12 & 0.04\end{array}\right].}\]

The following example demonstrates this computation in SciPy

    >>> A = mat('[1 3 5; 2 5 1; 2 3 8]')
    >>> A
    matrix([[1, 3, 5],
            [2, 5, 1],
            [2, 3, 8]])
    >>> A.I
    matrix([[-1.48,  0.36,  0.88],
            [ 0.56,  0.08, -0.36],
            [ 0.16, -0.12,  0.04]])
    >>> from scipy import linalg
    >>> linalg.inv(A)
    array([[-1.48,  0.36,  0.88],
           [ 0.56,  0.08, -0.36],
           [ 0.16, -0.12,  0.04]])
    
Solving linear system
^^^^^^^^^^^^^^^^^^^^^

Solving linear systems of equations is straightforward using the scipy
command :obj:`linalg.solve`. This command expects an input matrix and
a right-hand-side vector. The solution vector is then computed. An
option for entering a symmetrix matrix is offered which can speed up
the processing when applicable.  As an example, suppose it is desired
to solve the following simultaneous equations:

.. math::
   :nowrap:

    \begin{eqnarray*} x+3y+5z & = & 10\\ 2x+5y+z & = & 8\\ 2x+3y+8z & = & 3\end{eqnarray*}

We could find the solution vector using a matrix inverse:

.. math::
   :nowrap:

    \[ \left[\begin{array}{c} x\\ y\\ z\end{array}\right]=\left[\begin{array}{ccc} 1 & 3 & 5\\ 2 & 5 & 1\\ 2 & 3 & 8\end{array}\right]^{-1}\left[\begin{array}{c} 10\\ 8\\ 3\end{array}\right]=\frac{1}{25}\left[\begin{array}{c} -232\\ 129\\ 19\end{array}\right]=\left[\begin{array}{c} -9.28\\ 5.16\\ 0.76\end{array}\right].\]

However, it is better to use the linalg.solve command which can be
faster and more numerically stable. In this case it however gives the
same answer as shown in the following example:

    >>> A = mat('[1 3 5; 2 5 1; 2 3 8]')
    >>> b = mat('[10;8;3]')
    >>> A.I*b
    matrix([[-9.28],
            [ 5.16],
            [ 0.76]])
    >>> linalg.solve(A,b)
    array([[-9.28],
           [ 5.16],
           [ 0.76]])


Finding Determinant
^^^^^^^^^^^^^^^^^^^

The determinant of a square matrix :math:`\mathbf{A}` is often denoted
:math:`\left|\mathbf{A}\right|` and is a quantity often used in linear
algebra. Suppose :math:`a_{ij}` are the elements of the matrix
:math:`\mathbf{A}` and let :math:`M_{ij}=\left|\mathbf{A}_{ij}\right|`
be the determinant of the matrix left by removing the
:math:`i^{\textrm{th}}` row and :math:`j^{\textrm{th}}` column from
:math:`\mathbf{A}` . Then for any row :math:`i,`

.. math::
   :nowrap:

    \[ \left|\mathbf{A}\right|=\sum_{j}\left(-1\right)^{i+j}a_{ij}M_{ij}.\]

This is a recursive way to define the determinant where the base case
is defined by accepting that the determinant of a :math:`1\times1` matrix is the only matrix element. In SciPy the determinant can be
calculated with :obj:`linalg.det` . For example, the determinant of

.. math::
   :nowrap:

    \[ \mathbf{A=}\left[\begin{array}{ccc} 1 & 3 & 5\\ 2 & 5 & 1\\ 2 & 3 & 8\end{array}\right]\]

is

.. math::
   :nowrap:

    \begin{eqnarray*} \left|\mathbf{A}\right| & = & 1\left|\begin{array}{cc} 5 & 1\\ 3 & 8\end{array}\right|-3\left|\begin{array}{cc} 2 & 1\\ 2 & 8\end{array}\right|+5\left|\begin{array}{cc} 2 & 5\\ 2 & 3\end{array}\right|\\  & = & 1\left(5\cdot8-3\cdot1\right)-3\left(2\cdot8-2\cdot1\right)+5\left(2\cdot3-2\cdot5\right)=-25.\end{eqnarray*}

In SciPy this is computed as shown in this example:

    >>> A = mat('[1 3 5; 2 5 1; 2 3 8]')
    >>> linalg.det(A)
    -25.000000000000004


Computing norms
^^^^^^^^^^^^^^^

Matrix and vector norms can also be computed with SciPy. A wide range
of norm definitions are available using different parameters to the
order argument of :obj:`linalg.norm` . This function takes a rank-1
(vectors) or a rank-2 (matrices) array and an optional order argument
(default is 2). Based on these inputs a vector or matrix norm of the
requested order is computed.

For vector *x* , the order parameter can be any real number including
``inf`` or ``-inf``. The computed norm is

.. math::
   :nowrap:

    \[ \left\Vert \mathbf{x}\right\Vert =\left\{ \begin{array}{cc} \max\left|x_{i}\right| & \textrm{ord}=\textrm{inf}\\ \min\left|x_{i}\right| & \textrm{ord}=-\textrm{inf}\\ \left(\sum_{i}\left|x_{i}\right|^{\textrm{ord}}\right)^{1/\textrm{ord}} & \left|\textrm{ord}\right|<\infty.\end{array}\right.\]



For matrix :math:`\mathbf{A}` the only valid values for norm are :math:`\pm2,\pm1,` :math:`\pm` inf, and 'fro' (or 'f') Thus,

.. math::
   :nowrap:

    \[ \left\Vert \mathbf{A}\right\Vert =\left\{ \begin{array}{cc} \max_{i}\sum_{j}\left|a_{ij}\right| & \textrm{ord}=\textrm{inf}\\ \min_{i}\sum_{j}\left|a_{ij}\right| & \textrm{ord}=-\textrm{inf}\\ \max_{j}\sum_{i}\left|a_{ij}\right| & \textrm{ord}=1\\ \min_{j}\sum_{i}\left|a_{ij}\right| & \textrm{ord}=-1\\ \max\sigma_{i} & \textrm{ord}=2\\ \min\sigma_{i} & \textrm{ord}=-2\\ \sqrt{\textrm{trace}\left(\mathbf{A}^{H}\mathbf{A}\right)} & \textrm{ord}=\textrm{'fro'}\end{array}\right.\]

where :math:`\sigma_{i}` are the singular values of :math:`\mathbf{A}` .

Examples:

    >>> A=np.array([[1, 2],[3, 4]])
    >>> A
    array([[1, 2],
          [3, 4]])
    >>> linalg.norm(A)
    5.4772255750516612
    >>> linalg.norm(A,'fro') #frobenius norm is the default
    5.4772255750516612
    >>> linalg.norm(A,1) #L1-norm
    6
    >>> linalg.norm(A,-1)
    4
    >>> linalg.norm(A,inf)
    7


Solving linear least-squares problems and pseudo-inverses
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Linear least-squares problems occur in many branches of applied
mathematics. In this problem a set of linear scaling coefficients is
sought that allow a model to fit data. In particular it is assumed
that data :math:`y_{i}` is related to data :math:`\mathbf{x}_{i}`
through a set of coefficients :math:`c_{j}` and model functions
:math:`f_{j}\left(\mathbf{x}_{i}\right)` via the model

.. math::
{'fraction_non_alphanumeric': 0.19130813230539667, 'fraction_numerical': 0.04097239492663517, 'mean_word_length': 3.330910070005385, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 115, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4867309', 'n_tokens_mistral': 5976, 'n_tokens_neox': 5488, 'n_words': 1775}