<commit_before>
In the previous section we derived differential equations that model mechanical, electrical, and electro-mechanical systems.  The equations themselves often do not provide sufficient information.  For example, we were able to find a signal $p$ representing the position of the mass-spring-damper in Figure~\ref{mech:massspring1} given a particular force signal $f$ is applied to the mass.  However, it is not immediately obvious how to find the force signal $f$ given a particular position signal $p$.  We will be able to solve this problem and, more generally, to describe properties of systems modelled by linear differential equations with constant coefficient, if we make the added assumptions that the systems are \term{linear} and \term{shift-invariant}.  We study linear shift-invariant systems in this chapter.  Throughout this chapter $H$ will denote a linear shift-invariant system.

\section{Convolution, regular systems and the delta ``function''} \label{sec:conv-regul-syst}

A large number of linear shift-invariant systems can be represented by a signal called the \term{impulse response}.  The impulse response of a system $H$ is a locally integrable signal $h$ such that
\[
Hx(t) = \int_{-\infty}^{\infty} h(\tau) x(t - \tau) d\tau,
\]
that is, the response of $H$ to input signal $x$ can be represented as an integral equation involving $x$ and the impulse response $h$.  The integral is called a \term{convolution} and appears so often that a special notation is used for it.  We write $h * x$ to  indicate the signal that results from convolution of signals $h$ and $x$, that is, $h * x$ is the signal
\[
h * x = \int_{-\infty}^{\infty} h(\tau) x(t - \tau) d\tau.
\]
%where $(h*x)(t)$ indicates $h*x$ evaluated at $t$.  
Those systems that have an impulse response we call \term{regular systems}\footnote{The name \term{regular system} is motivated by the term \term{regular distribution}~\citep{Zemanian_dist_theory_1965}}.  Unless otherwise stated the domain of the regular system $H$ is assumed to be the set of signals $x$ such that the integral
\[
\int_{-\infty}^{\infty} \abs{h(\tau) x(t - \tau)} d\tau < \infty \qquad \text{for all $t \in \reals$}.
\]
This set of signals is denoted by $\fin(h)$ and so $H \in \fin(h) \to \reals \to \complex$.  Observe that for all $x \in \fin(h)$ the response 
\[
Hx(t) = \int_{-\infty}^{\infty} h(\tau) x(t - \tau) d\tau \leq \int_{-\infty}^{\infty} \abs{h(\tau) x(t - \tau)} d\tau < \infty 
\]
for all $t \in \reals$.  It can be shown that $\fin(h)$ is a linear shift-invariance space (Exercise~\ref{exer:finhlinshiftinv}).

Regular systems are linear because, for all $x, y \in \fin(h)$ and all $a, b \in \complex$,
\begin{equation}\begin{split}\label{eq:regsystemislinear}
H(ax + by) &= h * (ax + by) \\
&= \int_{-\infty}^{\infty} h(\tau) \big(ax(t - \tau) + by(t - \tau)\big) d\tau \\
&= a\int_{-\infty}^{\infty} h(\tau) x(t - \tau) d\tau + b\int_{-\infty}^{\infty} h(\tau)y(t - \tau) d\tau \\
&= a (h * x) + b(h*y) \\
&= a Hx + bHy.
\end{split}\end{equation}
The above equation shows that convolution commutes with scalar multiplication and distributes with addition, that is,
\[
h * (ax + by) =  a (h * x) + b(h*y).
\]
Regular systems are also shift-invariant because for all $x \in \fin(h)$
\begin{align*}
T_\kappa H x &= T_\kappa(h * x ) \\
&= \int_{-\infty}^{\infty} h(\tau) x(t- \kappa - \tau) d\tau \\
&= \int_{-\infty}^{\infty} h(\tau) T_\kappa x(t - \tau) d\tau \\ 
&= h * (T_\kappa x) \\
&= H T_\kappa x. 
\end{align*}

The impulse response of a regular system $H$ can be found in the following way.  First define the signal
\[
p_\gamma(t) = \begin{cases}
\gamma, & 0 < t \leq \frac{1}{\gamma} \\
0, & \text{otherwise},
\end{cases}
\]
that is, a rectangular shaped pulse of height $\gamma$ and width $\tfrac{1}{\gamma}$.  The signal $p_\gamma$ is plotted in Figure~\ref{fig:rectpulsedelta} for $\gamma=\frac{1}{2},1,2,5$.  As $\gamma$ increases the pulse gets thinner and higher so as to keep the area under $p_\gamma$ equal to one.  Consider the response of the regular system $H$ to the signal $p_\gamma$,
\begin{align*}
H p_\gamma = h * p_\gamma = \int_{-\infty}^\infty h(\tau) p_\gamma(t - \tau) d\tau = \gamma \int_{t- 1/\gamma}^{t} h(\tau) d\tau.
\end{align*}
Taking $\gamma \to \infty$ we find that
\[
\lim_{\gamma \rightarrow \infty} H p_\gamma = \lim_{\gamma \rightarrow \infty} \gamma \int_{t- 1/\gamma}^{t} h(\tau) d\tau = h \;\; \text{a.e.}.
\]
%that is, the impulse response satisfies $h = \lim_{\gamma \rightarrow \infty} H p_\gamma$ almost everywhere.  %If this limit does not converge for any $t$, then the system is not regular and does not have an impulse response.  %The reader unclear on the meaning of the limit above might want to review Section~\ref{sec:convergence-signals}.

%BLERG Young's inequality gives a neat way of knowing when convolutions exist or don't exists based on what Lp space the signals lie in.  This becomes useful in the Fourier transform section?

As an example, consider the integrator system $I_\infty$ described in Section~\ref{sec:some-import-syst}.  This response of $I_\infty$ to $p_\gamma$ is 
\[
I_\infty p_\gamma(t) = \int_{-\infty}^{t} p_\gamma(\tau) d\tau = \begin{cases}
0, & t \leq 0 \\
\gamma t, & 0 < t \leq \frac{1}{\gamma} \\
1, & t > \frac{1}{\gamma}
\end{cases}
\]
The response is plotted in Figure~\ref{fig:rectpulsedelta}.  Taking the limit as $\gamma \rightarrow\infty$ we find that the impulse response of the integrator is the step function
\begin{equation}\label{eq:utimprespstep}
u(t) = \lim_{\gamma\rightarrow\infty} H p_\gamma(t) = \begin{cases}
0 & t \leq 0 \\
1 & t > 0.
\end{cases} \qquad \text{a.e.}
\end{equation}

\newcommand{\rectpulse}[1]{\draw[color=black,thick] (-0.5,0) -- (0,0) -- (0,#1) -- (1.0/#1,#1) node[right] {$\gamma=#1$}-- (1.0/#1,0) -- (3,0)}
\newcommand{\pulseresponse}[2]{\draw[color=black,thick] (-0.5,0) -- (0,0) -- node [rotate=atan(#1),pos=#2,below] {$\gamma=#1$} (4*1.0/#1,4*1) -- (4*5,4*1)}
\begin{figure}[tp]
  \centering
  \begin{tikzpicture}
    % \draw[very thin,color=gray] (-0.1,-1.1) grid (3.9,3.9);
    \draw[->] (-0.75,0) -- (3.5,0) node[above] {$t$};
    \draw[->] (0,-0.5) -- (0,5.5) node[right] {$p_\gamma(t)$};
    \rectpulse{1.0};<commit_msg>minor<commit_after>
In the previous section we derived differential equations that model mechanical, electrical, and electro-mechanical systems.  The equations themselves often do not provide sufficient information.  For example, we were able to find a signal $p$ representing the position of the mass-spring-damper in Figure~\ref{mech:massspring1} given a particular force signal $f$ is applied to the mass.  However, it is not immediately obvious how to find the force signal $f$ given a particular position signal $p$.  We will be able to solve this problem and, more generally, to describe properties of systems modelled by linear differential equations with constant coefficient, if we make the added assumptions that the systems are \term{linear} and \term{shift-invariant}.  We study linear shift-invariant systems in this chapter.  %Throughout this chapter $H$ will denote a linear shift-invariant system.

\section{Convolution, regular systems and the delta ``function''} \label{sec:conv-regul-syst}

A large number of linear shift-invariant systems can be represented by a signal called the \term{impulse response}.  The impulse response of a system $H$ is a locally integrable signal $h$ such that
\[
Hx(t) = \int_{-\infty}^{\infty} h(\tau) x(t - \tau) d\tau,
\]
that is, the response of $H$ to input signal $x$ can be represented as an integral equation involving $x$ and the impulse response $h$.  The integral is called a \term{convolution} and appears so often that a special notation is used for it.  We write $h * x$ to  indicate the signal that results from convolution of signals $h$ and $x$, that is, $h * x$ is the signal
\[
h * x = \int_{-\infty}^{\infty} h(\tau) x(t - \tau) d\tau.
\]
%where $(h*x)(t)$ indicates $h*x$ evaluated at $t$.  
Those systems that have an impulse response we call \term{regular systems}\footnote{The name \term{regular system} is motivated by the term \term{regular distribution}~\citep{Zemanian_dist_theory_1965}}.  Unless otherwise stated the domain of the regular system $H$ is assumed to be the set of signals $x$ such that the integral
\[
\int_{-\infty}^{\infty} \abs{h(\tau) x(t - \tau)} d\tau < \infty \qquad \text{for all $t \in \reals$}.
\]
This set of signals is denoted by $\fin h$ and so $H \in \fin h \to \reals \to \complex$.  % Observe that for all $x \in \fin h$ the response 
% \[
% Hx(t) = \int_{-\infty}^{\infty} h(\tau) x(t - \tau) d\tau \leq \int_{-\infty}^{\infty} \abs{h(\tau) x(t - \tau)} d\tau < \infty 
% \]
% for all $t \in \reals$.  
It can be shown that $\fin h$ is a linear shift-invariance space (Exercise~\ref{exer:finhlinshiftinv}).

Regular systems are linear because, for all $x, y \in \fin h$ and all $a, b \in \complex$,
\begin{equation}\begin{split}\label{eq:regsystemislinear}
H(ax + by) &= h * (ax + by) \\
&= \int_{-\infty}^{\infty} h(\tau) \big(ax(t - \tau) + by(t - \tau)\big) d\tau \\
&= a\int_{-\infty}^{\infty} h(\tau) x(t - \tau) d\tau + b\int_{-\infty}^{\infty} h(\tau)y(t - \tau) d\tau \\
&= a (h * x) + b(h*y) \\
&= a Hx + bHy.
\end{split}\end{equation}
The above equation shows that convolution commutes with scalar multiplication and distributes with addition, that is,
\[
h * (ax + by) =  a (h * x) + b(h*y).
\]
Regular systems are also shift-invariant because for all $x \in \fin h$
\begin{align*}
T_\kappa H x &= T_\kappa(h * x ) \\
&= \int_{-\infty}^{\infty} h(\tau) x(t- \kappa - \tau) d\tau \\
&= \int_{-\infty}^{\infty} h(\tau) T_\kappa x(t - \tau) d\tau \\ 
&= h * (T_\kappa x) \\
&= H T_\kappa x. 
\end{align*}

The impulse response of a regular system $H$ can be found in the following way.  First define the signal
\[
p_\gamma(t) = \begin{cases}
\gamma, & 0 < t \leq \frac{1}{\gamma} \\
0, & \text{otherwise},
\end{cases}
\]
that is, a rectangular shaped pulse of height $\gamma$ and width $\tfrac{1}{\gamma}$.  The signal $p_\gamma$ is plotted in Figure~\ref{fig:rectpulsedelta} for $\gamma=\frac{1}{2},1,2,5$.  As $\gamma$ increases the pulse gets thinner and higher so as to keep the area under $p_\gamma$ equal to one.  Consider the response of the regular system $H$ to the signal $p_\gamma$,
\begin{align*}
H p_\gamma = h * p_\gamma = \int_{-\infty}^\infty h(\tau) p_\gamma(t - \tau) d\tau = \gamma \int_{t- 1/\gamma}^{t} h(\tau) d\tau.
\end{align*}
Taking $\gamma \to \infty$ we find that
\[
\lim_{\gamma \rightarrow \infty} H p_\gamma = \lim_{\gamma \rightarrow \infty} \gamma \int_{t- 1/\gamma}^{t} h(\tau) d\tau = h \;\; \text{a.e.}
\]
%that is, the impulse response satisfies $h = \lim_{\gamma \rightarrow \infty} H p_\gamma$ almost everywhere.  %If this limit does not converge for any $t$, then the system is not regular and does not have an impulse response.  %The reader unclear on the meaning of the limit above might want to review Section~\ref{sec:convergence-signals}.

%BLERG Young's inequality gives a neat way of knowing when convolutions exist or don't exists based on what Lp space the signals lie in.  This becomes useful in the Fourier transform section?

As an example, consider the integrator system $I_\infty$ described in Section~\ref{sec:some-import-syst}.  The response of $I_\infty$ to $p_\gamma$ is 
\[
I_\infty p_\gamma(t) = \int_{-\infty}^{t} p_\gamma(\tau) d\tau = \begin{cases}
0, & t \leq 0 \\
\gamma t, & 0 < t \leq \frac{1}{\gamma} \\
1, & t > \frac{1}{\gamma}.
\end{cases}
\]
The response is plotted in Figure~\ref{fig:rectpulsedelta}.  Taking the limit as $\gamma \rightarrow\infty$ we find that the impulse response of the integrator is the step function
\begin{equation}\label{eq:utimprespstep}
u(t) = \lim_{\gamma\rightarrow\infty} H p_\gamma(t) = \begin{cases}
0 & t \leq 0 \\
1 & t > 0.
\end{cases} \qquad \text{a.e.}
\end{equation}

\newcommand{\rectpulse}[1]{\draw[color=black,thick] (-0.5,0) -- (0,0) -- (0,#1) -- (1.0/#1,#1) node[right] {$\gamma=#1$}-- (1.0/#1,0) -- (3,0)}
\newcommand{\pulseresponse}[2]{\draw[color=black,thick] (-0.5,0) -- (0,0) -- node [rotate=atan(#1),pos=#2,below] {$\gamma=#1$} (4*1.0/#1,4*1) -- (4*5,4*1)}
\begin{figure}[tp]
  \centering
  \begin{tikzpicture}
    % \draw[very thin,color=gray] (-0.1,-1.1) grid (3.9,3.9);
    \draw[->] (-0.75,0) -- (3.5,0) node[above] {$t$};
    \draw[->] (0,-0.5) -- (0,5.5) node[right] {$p_\gamma(t)$};
    \rectpulse{1.0};
{'fraction_non_alphanumeric': 0.16881350508040643, 'fraction_numerical': 0.013921113689095127, 'mean_word_length': 3.6108447067502767, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5435229', 'n_tokens_mistral': 4305, 'n_tokens_neox': 4157, 'n_words': 1587}