text:
A Chat With July's Profile, Doug Mockett, founder/owner of Doug Mockett & Co.
Profile of Success, Catalog Success magazine, July 2008
By Joe Keenan
Catalog Success: Where's the company headquartered?
Doug Mockett: It's physically located in Torrance, California.
CS: When was the company established?
DM: In early 1981.
CS: When were the first catalogs mailed?
DM: We got into the mailing of the catalog, it had to be about two years later. It wasn't so much a catalog as it was a single-sheet flyer; mainly because I only had one product at the time and had no money.
CS: What was the one product that you were selling at the time?
DM: Grommets, wire access grommets. We just had basically two different sizes and two different colors.
CS: What are your customer demographics?
DM: Well, our customers are major office furniture manufacturers, they're large- and mid-sized mill workshops, which are the people who do custom interiors for banks, law firms, board rooms, audio/visual, that type of stuff. And then a whole host of little, small cabinet shops. And then we have another huge market that we don't sell to but they specify our parts to go into the furnishings of the buildings they're designing, and that's architects and designers.
CS: Do you strictly sell B-to-B?
DM: Not really. I mean, we sell to anyone who calls and places an order, but it's 99 and nine-tenths percent B-to-B.
CS: What's the primary merchandise you offer?
DM: The main thing we offer and sell are components for office and computer furniture. And that consists of table legs; drawer pulls; innovative components to integrate computers into furniture; a whole line of wire access grommets and wire management, which are the channels to channel wires after they go through the hole in the desk. And the last few years we've added a line of components for the kitchen, bath and closet. It is, I think, at the moment 2,500 SKUs.
1 2 3 4 5 6 7 8 9 AllNext »
E Joe Keenan Author's page
Joe Keenan is the executive editor of Total Retail. Joe has more than 10 years experience covering the retail industry, and enjoys profiling innovative companies and people in the space.
Amazon Shuts Down Amazon Pantry
Top Tips for Retailers to Kick-Start 2021
J.C. Penney's Owners Launch Search for New CEO
25 Best Tips of 2020
A Review of Total Retail's Top Content in 2020

redpajama_set_name: RedPajamaCommonCrawl


text:
Compromised Administrative Credentials Lead to $2.3 Million Settlement With HIPAA Business Associate
Compromised Administrative Credentials Lead to $2.3 Million Settlement With HIPAA Business Associate HHS Resolution Agreement: CHSPSC LLC (March 30, 2020); HHS News Release (Sept. 23, 2020)
Available at https://www.hhs.gov/hipaa/for-professionals/compliance-enforcement/agreements/chspsc/index.html
HHS's Office for Civil Rights (OCR) has announced a $2.3 million settlement to resolve potential violations of HIPAA's privacy and security rules related to a breach of protected health information (PHI) affecting more than six million individuals. The breach occurred when a cyberhacking group used compromised administrative credentials to remotely access, through a virtual private network, the information system maintained by a HIPAA business associate. (The business associate provided IT and health information management services to hospitals and physician clinics.) According to OCR, the business associate was unaware of the intrusion until notified by the FBI that an advanced persistent threat had been traced to the business associate's information system. Despite the FBI's notice, the hackers were able to exfiltrate PHI—including names, birthdates, contact information, and Social Security numbers—from the information system for several more months. OCR's investigation uncovered "longstanding, systemic noncompliance" with the HIPAA security rule, including failures to conduct a risk analysis and to implement procedures for information system activity review, security incident response and mitigation, and access controls.
In addition to the settlement payment, the business associate agreed to an extensive corrective action plan (CAP). The CAP requires the business associate to conduct an enterprise-wide risk analysis and adopt a corresponding risk management plan, each subject to OCR review and approval. The business associate must also revise—subject to OCR approval—numerous security policies and procedures, including those addressing technical access controls for software applications and network or server equipment; activity logs; security incident response, reporting, and mitigation; and password management, strength, and safeguarding. The approved policies and procedures must be incorporated into training materials that must be included in training sessions for all workforce members to be held within 30 days of OCR's approval of the materials. New workforce members must be trained within 14 days after they start work. The business associate also must develop—subject to OCR approval—a plan to internally monitor compliance with the CAP and implement the plan for a specified period, during which periodic compliance reports must be submitted to OCR.
EBIA Comment: As we have noted previously (see our Checkpoint article), security alerts from law enforcement agencies should trigger immediate action—but immediate action is feasible only if covered entities and business associates plan ahead. Robust procedures for tracking information system activity and responding to security incidents are required under the HIPAA security rule and can help contain breaches and mitigate damages. It's always better to identify and respond to security incidents before the G-men come knocking. For more information, see EBIA's HIPAA Portability, Privacy & Security manual at Sections XX.D ("Resolution Agreements"), XXV.H ("Guide to Planning for Breach Notification"), XXX.B.1.d ("Information System Activity Review (Required Implementation Specification)"), and XXX.B.6 ("Standard: Security Incident Procedures"). You may also be interested in our webinar "HIPAA Breaches: Preparation and Response" (recorded on 9/10/20).
New WCO product classifications are here. Are companies ready?
The Tax Technology Transformation: Getting Started
November 23, 2022 · 5 minute read
Understanding Clean Vehicle Credits for Electric Vehicles
November 9, 2022 · 6 minute read


I just had the opportunity to listen to my favorite song by the Association, "Everything That Touches You." While it wasn't their biggest hit by a long shot, it was their best song (in my less than humble opinion). Enjoy!
Sounds like the Beach Boys and The Mama's and the Papa's had a baby and they called it pleasant schlock!
Yes, that's sort of true. However, I will never be 14 again and at 14, this song was cool. I'm guessing that James West and Artemus Gordon thought it was cool, Ilya Kuriakin and Napoleon Solo thought it was cool, etc. Also, the song is terrifically hook laden and the Wrecking Crew laid down a nice, mid 60s, LA groove.
On those rare occasions when I hear stuff like this, I am always knocked out again with the memory of how cool this kind of music sounded to me. You know, it was in that brief time from about 1965 to 1967-68 when the eastern doo-wop had turned into the Four Seasons and the LA cats were bringing their laid-back, folk/rock approach to that sound. The Association, Turtles, Mamas and Papas, Beach Boys, and others all worked in this vein before it faded away in 1968-69.
Anyway, no amount of snarky comments shall vanquish my affection for songs like this. Some songs matter just 'cause they take you back to that place long ago. That's one of the great things about music--even (or perhaps especially) this music.
Well Artie, I guess you're right, its still fairly gay though, but it's a well crafted sort of gayness. Yes, it reminds me of the days of yore when guys in bands had those page boy haircuts and they were by default better qualified to be jockeys than say....actual men. But that's just my opinion. Excuse me while I go listen to some Sinatra or maybe some Wilson Pickett or better still, I think I'll sing me a song, "boys and things they come by the dozen, that ain't nuthin but 10 cent luvin, hey pretty mama let light your candle cause mama I'm to hard to handle ,yes I am"! By the way, I wasn't singing that to you, so don't get any ideas. You never know about people who listen to certain kinds of music!
Otis Wannabe,Well, one thing is true, even though you didn't mention it. With the possible exception of the Lovin' Spoonful, has anyone ever made a recording with an autoharp that didn't sound gay/feminine? And this song certainly has its share of auto-harp.
That said, I still like it, which, I guess, makes me a manly man who is in touch with his feelings and not some overcompensating, insecure macho man like, well, you know who I'm talkin' about. Yes, I'm the kind of strong, sensitive man that women want and need.
I take it back. It isn't a combination of The Beach Boys and The Mama's and The Papa's, its more like The Pointer Sisters meets The San Francisco Gay Men's Choir, "and all the colored girls go doo, do, doo, do, doo, do, doo, do!"
If this conversation has been offensive in any way, we sincerely apologize. In fact, we don't know who these people are, nor do we condone their use of pejorative terms like "Gay" or support the idea that "The Association" ever made good records! There are clearly some who would disagree--and for that we apologize as well!


text:
\section{Introduction}

Gaussian process (GP) is a non-parametric supervised learning technique that estimates a posterior distribution of predictors from the dataset~\cite{Rasmussen2005}. In this study, we consider meta-learning for GP. Meta-learning is a framework used to improve the precision of new tasks by estimating a structure of a set of tasks~\cite{Vilalta2002,Hospedales2020,Huisman2021}. Most conventional meta-learning methods for GP estimate a prior distribution for new tasks~\cite{Anton2005,Bonilla2007,Li2018,Fortuin2020}. We propose an extension of principal component analysis for a set of Gaussian process posteriors (GP-PCA) to estimate a low-dimensional subspace on a space of GP posteriors. Since GP-PCA estimates a subspace on a space of GPs, the method can generate GP posteriors for new tasks. Therefore, we can estimate the GP posterior for the new tasks accurately from a small size of the dataset.

To this end, we have to consider a space of GPs with an infinite-dimensional parameter. A structure of a probability space is nontrivial since Euclidean space is inappropriate as a structure of the space. For a finite parametric probability distribution, we can define a structure of its space using information geometry~\cite{Amari2016}. However, even if we use the information geometry, it is not easy to define a space of GPs.

To overcome this problem, we consider defining the space of GP posteriors under the assumption that GP posteriors have the same prior. Then, we can show that the set of GP posteriors lies on a finite-dimensional subspace in an infinite-dimensional space of GP. By using this fact, we can reduce the task of GP-PCA to a task of estimating a subspace on finite-dimensional space. Additionally, we developed a fast approximation method for GP-PCA using a sparse GP based on variational inference.

The remainder of the paper is organized as follows. In Section~2, we explain the information geometry, principal component analysis for exponential families and GP regression. In Section~3, after defining a set of GP posteriors in terms of information geometry, we propose the GP-PCA and show that the task can be reduced to a finite-dimensional case. In Section~4, we present the related works. In Section~5, we demonstrate the effectiveness of the proposed method. Finally, Section~6 presents the conclusion.

\section{Preliminaries}
In this section, we explain the information geometry of the exponential family, dimensionality reduction technique on the exponential family, and GP.
\subsection{Information geometry of the exponential family}
The exponential family is a distribution parameterized by $\boldsymbol{\xi}=(\xi_1,\xi_2,\cdots,\xi_D)$ as follows.
\begin{align*}
    p(x\mid \boldsymbol{\xi})=\exp(\boldsymbol{\xi}^{\rm T} \mathbf{G}(x) + C(x) - \psi(\boldsymbol{\xi})).
\end{align*}
In information geometry, a set of $p(x\mid \boldsymbol{\xi})$ is regarded as a Riemannian manifold denoted by $\mathcal{S}$. Then, a metric of $\mathcal{S}$ is defined by Fisher information
\begin{equation*}
    g_{ij}(\boldsymbol{\xi})=E_{p(x\mid \boldsymbol{\xi})}\left[\left(\frac{\partial}{\partial \xi_i}\log{p(x\mid \boldsymbol{\xi})}\right)\left(\frac{\partial}{\partial \xi_j}\log{p(x\mid \boldsymbol{\xi})}\right)\right],
\end{equation*}
and a connection is defined by $\alpha$-connection, where $\alpha \in \mathbb{R}$ is a parameter of $\alpha$-connection. When $\alpha=\pm1$, $\mathcal{S}$ can be regarded as a flat manifold, i.e., curvature and torsion of $\mathcal{S}$ are zero. When $\alpha=1$, $\mathcal{S}$ is a flat manifold defined in a coordinate system by $\boldsymbol{\xi}$, which is called e-coordinate system. On the other hand, when $\alpha=-1$, $\mathcal{S}$ is a flat manifold defined in a coordinate system by $\boldsymbol{\zeta}:=E_{p(x\mid \boldsymbol{\xi})}[\mathbf{G}(x)]$, which is called m-coordinate system.

There is a bijection between $\boldsymbol{\xi}$ and $\boldsymbol{\zeta}$, and the bijection can be described as Legendre transform. The following equation with respect to $\boldsymbol{\xi}$ and $\boldsymbol{\zeta}$ holds.
\begin{equation*}
    \psi(\boldsymbol{\xi})+\phi(\boldsymbol{\zeta})-\boldsymbol{\xi}^{\rm T}\boldsymbol{\zeta} = 0,
\end{equation*}
where $\psi(\boldsymbol{\xi})$ and $\phi(\boldsymbol{\zeta})$ are potential functions of $\boldsymbol{\xi}$ and $\boldsymbol{\zeta}$, respectively. From the equation, $\boldsymbol{\xi}$ and $\boldsymbol{\zeta}$ can be mutually transformed by Legendre transformation as follows.
\begin{align*}
    \frac{\partial \psi(\boldsymbol{\xi})}{\partial \boldsymbol{\xi}}&=\boldsymbol{\zeta}, \\
    \frac{\partial \phi(\boldsymbol{\zeta})}{\partial \boldsymbol{\zeta}}&=\boldsymbol{\xi}.
\end{align*}
From this fact, $\boldsymbol{\xi}$ and $\boldsymbol{\zeta}$ are in a nonlinear relationship in general. Therefore, e-flat manifold does not always become m-flat manifold and vice versa. If a manifold becomes e-flat and m-flat simultaneously, the manifold is called a dually flat manifold. Since $\mathcal{S}$ holds e-flat and m-flat, $\mathcal{S}$ is a dually flat manifold.

In a dually flat manifold, we can consider two kinds of linear subspaces: e-flat and m-flat subspaces. Let $\boldsymbol{\xi}_i$ and $\boldsymbol{\zeta}_i$ be an e-coordinate and m-coordinate of $p_i \in \mathcal{S}$. While an e-flat subspace is defined as a linear combination of $\Xi=\{\boldsymbol{\xi}_i\}^I_{i=1}$, an m-flat subspace is defined as a linear combination of $Z=\{\boldsymbol{\zeta}_i\}^I_{i=1}$. Let $\mathcal{M}_e$ and $\mathcal{M}_m$ be e-flat and m-flat subspaces, respectively. Then, $\mathcal{M}_e$ and $\mathcal{M}_m$ are described as follows.
\begin{equation}
    \mathcal{M}_e=\Set{ \boldsymbol{\xi}(\mathbf{t},\Xi) = \sum^M_{m=1}t_i\boldsymbol{\xi}_m | \sum^M_{m=1}t_m=1 },
\end{equation}
\begin{equation}
    \mathcal{M}_m=\Set{ \boldsymbol{\zeta}(\mathbf{t},Z) = \sum^M_{m=1}t_m\boldsymbol{\zeta}_m | \sum^M_{m=1}t_m=1 },
\end{equation}
where $\mathbf{t}=(t_1,t_2,\cdots,t_M)$. When $M=2$, $\mathcal{M}_e$ and $\mathcal{M}_m$ are called e-geodesic and m-geodesic, respectively.

By using $\boldsymbol{\xi}_i$ and $\boldsymbol{\zeta}_i$, we define a Kullback-Leibler (KL) divergence between the two points $p_i,p_j \in \mathcal{S}$ as follows.
\begin{equation}
    \KL{p_i}{p_j}=\psi(\boldsymbol{\xi}_i)+\phi(\boldsymbol{\zeta}_j)-\boldsymbol{\xi}^T_i\boldsymbol{\zeta}_j.
\end{equation}
We denote the KL divergence by using e-coordinates or m-coordinates depending on the situation, i.e., $\KL{\boldsymbol{\xi}_i}{\boldsymbol{\xi}_j}$ and $\KL{\boldsymbol{\zeta}_i}{\boldsymbol{\zeta}_j}$. The following theorems show an interesting duality of e-coordinate and m-coordinate.
\begin{theorem}[Pythagorean theorem~\cite{Amari2016}]
Let $p_i$, $p_j$ and $p_k$ be points on $\mathcal{S}$. If an e-geodesic between $p_i$ and $p_j$ and an m-geodesic between $p_j$ and $p_k$ are orthogonal, i.e., $(\boldsymbol{\xi}_i-\boldsymbol{\xi}_j)^{\rm T}(\boldsymbol{\zeta}_j-\boldsymbol{\zeta}_k)=0$ holds. Then, the following relationship holds.
\begin{equation*}
    \KL{p_i}{p_k} = \KL{p_i}{p_j} + \KL{p_j}{p_k}.
\end{equation*}
\end{theorem}

When an m-geodesic between $p \in \mathcal{S}$ and $q \in \mathcal{M}_e \subset \mathcal{S}$ are orthogonal, $q$ is called m-projection from $p$ to $\mathcal{M}_e$. Similarly, when an e-geodesic between $p \in \mathcal{S}$ and $q \in \mathcal{M}_m \subset \mathcal{S}$ are orthogonal, $q$ is called e-projection from $p$ to $\mathcal{M}_m$. From the Pythagorean theorem, the following theorem holds.
\begin{theorem}[Projection theorem\cite{Amari2016}]
An m-projection from $p \in \mathcal{S}$ to $q \in \mathcal{M}_e$ uniquely exists and it minimizes $\KL{p}{q}$. Similarly, an e-projection from $p \in \mathcal{S}$ to $q \in \mathcal{M}_m$ uniquely exists and it minimizes $\KL{q}{p}$.
\end{theorem}

Based on these properties, Principal Component Analysis (PCA) for exponential families have been proposed by \cite{Collins2001,Akaho2004}. We explain the method below.

\subsection{PCA for exponential families}

Let $\mathcal{S}$ be a set of exponential families. Since there are two types of subspaces on $\mathcal{S}$: e-flat and m-flat subspaces, we can consider two PCAs for a dataset $\mathcal{P}=\{p_1,p_2,\cdots,p_I\}\in \mathcal{S}$. One is e-PCA, which estimates an e-flat affine subspace $\mathcal{M}_e$. The other is m-PCA, which estimates an m-flat affine subspace $\mathcal{M}_m$. Although we only explain e-PCA, the same argument holds for m-PCA.

Assume that $\mathcal{M}_e$ can be described by $L$ basis $\{\mathbf{u}_1, \mathbf{u}_2, \ldots, \mathbf{u}_L\}$ and offset $\mathbf{u}_0$, where $\mathbf{u}_l \in \mathcal{S}, (l=0,1,\ldots,L)$. It means that using a weight vector $\mathbf{w}=(w_1, w_2, \ldots, w_L)^{\rm T}$ and basis $\mathbf{U}=(\mathbf{u}_0,\mathbf{u}_1, \mathbf{u}_2, \ldots, \mathbf{u}_L)^{\rm T}$, any point on $\mathcal{M}_e$ can be represented as
\begin{align*}
    \boldsymbol{\xi}(\mathbf{w}, \mathbf{U}) &= \sum^L_{l=1} w_l \mathbf{u}_l + \mathbf{u}_0 \\
    &=(1,\mathbf{w}^{\rm T}) \mathbf{U}.
\end{align*}

When $\{\boldsymbol{\xi}_i\}^I_{i=1}$ is obtained, the task of e-PCA is to estimate $\mathbf{W}=(\mathbf{w}_1,\mathbf{w}_2,\ldots,\mathbf{w}_I)^{\rm T}$ and $\mathbf{U}$ minimizing the following objective function.
\begin{align}
    E(\mathbf{W},\mathbf{U})=\sum^I_{i=1}D_{KL}[\boldsymbol{\xi}_i || \boldsymbol{\xi}(\mathbf{w}_i, \mathbf{U})].
    \label{objective_function_epca}
\end{align}
Because $\mathbf{W}$ and $\mathbf{U}$ minimizing $E(\mathbf{W},\mathbf{U})$ cannot be obtained analytically in general, e-PCA alternatively estimates $\mathbf{W}$ and $\mathbf{U}$ using a gradient method. Let $\boldsymbol{\zeta}_i$ and $\tilde{\boldsymbol{\zeta}}_i$ be m-coordinates of $\boldsymbol{\xi}_i$ and $\boldsymbol{\xi}(\mathbf{w}_i,\mathbf{U})$, respectively. We denote matrices of $\{\boldsymbol{\zeta}_i\}^I_{i=1}$ and $\{\tilde{\boldsymbol{\zeta}}_i\}^I_{i=1}$ by $\mathbf{Z}$ and $\tilde{\mathbf{Z}}$, respectively. The gradients of Eq.~\eqref{objective_function_epca} with respect to $\mathbf{W}$ and $\mathbf{U}$ are given by the following equations.
\begin{align}
    \frac{\partial E(\mathbf{W},\mathbf{U})}{\partial \mathbf{W}}&=(\hat{\mathbf{Z}}-\mathbf{Z})\tilde{\mathbf{U}}^{\rm T}, \\
    \frac{\partial E(\mathbf{W},\mathbf{U})}{\partial \mathbf{U}}&=\mathbf{W}^{\rm T}(\hat{\mathbf{Z}}-\mathbf{Z}),
    \label{delivation_objective_function_epca}
\end{align}
where $\tilde{\mathbf{U}}=(\mathbf{u}_1, \mathbf{u}_2, \ldots, \mathbf{u}_L)^{\rm T}$.

For multivariate normal distributions, each probabilistic distribution can be parameterized a mean vector $\boldsymbol{\mu}$ and covariance matrix $\mathbf{\Sigma}$. Letting $G_1(\mathbf{x})=\mathbf{x}$ and $G_2(\mathbf{x})= \mathbf{x} \mathbf{x}^{\rm T}$, the e-coordinate $\boldsymbol{\xi}$ can be described as follows:
\begin{align}
    \boldsymbol{\xi} &= (\boldsymbol{\theta}^{\rm T},{\rm vec}(\mathbf{\Theta})^{\rm T})^{\rm T},
\end{align}
where $\boldsymbol{\theta}=\mathbf{\Sigma}^{-1}\boldsymbol{\mu}$, $\mathbf{\Theta}=-\frac{1}{2}\mathbf{\Sigma}^{-1}$. On the other hand, the m-coordinate can be described as follows:
\begin{align}
    \boldsymbol{\zeta}=(\boldsymbol{\eta}^{\rm T},{\rm vec}(\mathbf{H})^{\rm T})^{\rm T},
\end{align}
where $\boldsymbol{\eta}= \boldsymbol{\mu}$, $\mathbf{H} = \boldsymbol{\mu} \boldsymbol{\mu}^{\rm T} + \mathbf{\Sigma}$.
Then, the transform between $\boldsymbol{\xi}$ and $\boldsymbol{\zeta}$ can be described as follows:
\begin{align*}
    \boldsymbol{\theta} &= \left(\mathbf{H} - \boldsymbol{\eta}\veta^{\rm T} \right)^{-1}\boldsymbol{\eta}, \\
    \mathbf{\Theta} &= -\frac{1}{2}\left(\mathbf{H} - \boldsymbol{\eta}\veta^{\rm T}\right)^{-1}, \\
    \boldsymbol{\eta} &= -\frac{1}{2}\mathbf{\Theta}^{-1}\boldsymbol{\theta}, \\
    \mathbf{H} &= \frac{1}{4}\mathbf{\Theta}^{-1}\boldsymbol{\theta}\vtheta^{\rm T}\mathbf{\Theta}^{-1} -\frac{1}{2}\mathbf{\Theta}^{-1}.
\end{align*}

\subsection{Gaussian process (GP)}
First, we present the definition of notations. An output vector of function $f$ corresponding to input set $X=\{x_n\}^N_{n=1}$ is denoted by $\mathbf{f}$ or $f(X)$. When an input set is denoted with a subscript, such as $X_A$, the corresponding output vector is also denoted with the subscript such as $\mathbf{f}_A$. Similarly, while a vector of kernel $k(x,x')$ between $X$ and $x$ is denoted by $\mathbf{k}(x):=k(X,x)$, a gram matrix between $X$ and $X$ is denoted by $\mathbf{K}:=k(X,X)$. The treatment of the subscript is the same as a function.

GP is a stochastic process with respect to a function $f:\mathcal{X} \rightarrow \mathbb{R}$. It is parameterized by the mean function $\mu:\mathcal{X} \rightarrow \mathbb{R}$ and covariance function $\sigma:\mathcal{X}\times\mathcal{X} \rightarrow \mathbb{R}$. The GP has a marginalization property. It means that a vector $\mathbf{f}=f(X)$ corresponding to an arbitrary input set $X$ can consistently follow a multivariate normal distribution $\mathbf{f} \sim \mathcal{N}(\boldsymbol{\mu},\mathbf{\Sigma})$, where $\boldsymbol{\mu}:=\mu(X)$ and $\mathbf{\Sigma}:=\sigma(X,X)$. Therefore, GP can be regarded as an infinite-dimensional multivariate normal distribution intuitively.

\subsubsection{Gaussian process regression (GPR)}
Let $x \in \mathcal{X}$ and $y \in \mathbb{R}$ be an input vector and output, respectively. We assume that the relationship between $x \in \mathcal{X}$ and $y \in \mathbb{R}$ is denoted as $y = f(x) + \varepsilon$, where $\varepsilon$ is a noise. The task of regression is to estimate a function $f:\mathcal{X} \rightarrow \mathbb{R}$ from an input set $X=\{x_n\}^N_{n=1}$ and corresponding output vector $\mathbf{y}=(y_1,y_2,\ldots,y_N)$. We assume that a likelihood function is a Gaussian distribution with mean $\mathbf{f}$ and variance $\beta^{-1}\mathbf{I}$, and the prior distribution is a GP with a mean function $\mu_0$ and covariance function $k$. For any $x_+$, $p(f(x_+),\mathbf{y})$ is obtained as
\begin{equation*}
  \left[
    \begin{array}{c}
      \mathbf{y}  \\
      f(x_+)
    \end{array}
  \right]
  \sim \mathcal{N}\left(
  \left[
    \begin{array}{c}
      \boldsymbol{\mu}_0  \\
      \mu_0(x_+)
\end{array}
  \right]
  ,
  \left[
    \begin{array}{cc}
      \mathbf{K}+\beta^{-1}\mathbf{I} & \mathbf{k}(x_+) \\
      \mathbf{k}^{\rm T}(x_+) & k(x_+,x_+)
    \end{array}
  \right]\right).
\end{equation*}
Since the posterior distribution is a conditional distribution of $f(x_+)$ given $\mathbf{y}$, the mean and covariance function for a new input $x$ of the posterior distribution can be obtained by closed form as $p(f(x_+) \mid \mathbf{y}) = \mathcal{N}(f(x_+) \mid \mu(x_+),\sigma(x_+,x_+))$, where
\begin{align}
    \mu(x_+)&=\mu_0(x_+)+\mathbf{k}^{\rm T}(x_+)\left(\mathbf{K}+\beta^{-1}\mathbf{I}\right)^{-1}(\mathbf{y}-\boldsymbol{\mu}_0), \label{eq_gp_posterior_mean} \\
    \sigma(x_+,x_+)&=k(x_+,x_+)-\mathbf{k}^{\rm T}(x_+)\left(\mathbf{K}+\beta^{-1}\mathbf{I}\right)^{-1}\mathbf{k}(x_+). \label{eq_gp_posterior_cov}
\end{align}

We can also interpret that the posterior is obtained using Bayes' theorem. When $X$ and $\mathbf{y}$ are observed, the posterior for $\mathbf{f}=f(X)$ is derived as follows:
\begin{equation*}
    q(\mathbf{f}\mid\mathbf{y}) = \frac{p(\mathbf{y}\mid\mathbf{f})p(\mathbf{f})}{p(\mathbf{y})}.
\end{equation*}
By using $q(\mathbf{f}\mid\mathbf{y})$, the predictive distribution for new input data $x_+$ is described as follows:
\begin{equation*}
    q(f(x_+)\mid\mathbf{y}) = \int p(f(x_+)\mid\mathbf{f})p(\mathbf{f}\mid\mathbf{y})d\mathbf{f},
    \label{eq_int_pos}
\end{equation*}
where $p(f(x_+)\mid\mathbf{f})$ is a conditional prior. Letting $q(\mathbf{f}\mid\mathbf{y})=\mathcal{N}(\boldsymbol{\mu},\mathbf{\Sigma})$, $\boldsymbol{\mu}$ and $\mathbf{\Sigma}$ are obtained as follows:
\begin{align*}
    \boldsymbol{\mu}&=\boldsymbol{\mu}_0+\mathbf{K}\left(\mathbf{K}+\beta^{-1}\mathbf{I}\right)^{-1}(\mathbf{y}-\boldsymbol{\mu}_0),\\
    \mathbf{\Sigma}&=\mathbf{K}-\mathbf{K}\left(\mathbf{K}+\beta^{-1}\mathbf{I}\right)^{-1}\mathbf{K}.
\end{align*}
Furthermore, the predictive distribution is derived as
\begin{align*}
    q(f(x_+)\mid\mathbf{y})=&\mathcal{N}(f(x_+)\mid \mu(x_+),\sigma(x_+,x_+)) \\
    \mu(x_+)&=\mu_0(x_+)+\mathbf{k}^{\rm T}(x_+)\mathbf{K}^{-1}\boldsymbol{\mu}, \\
    \sigma(x_+,x_+)&=k(x_+,x_+)+\mathbf{k}^{\rm T}(x_+)\mathbf{K}^{-1}(\mathbf{\Sigma}-\mathbf{K})\mathbf{K}^{-1}\mathbf{k}(x_+).
\end{align*}
Since $p(f(x_+)\mid\mathbf{f})$ is a prior distribution, $q(f(x_+)\mid\mathbf{y})$ is determined uniquely when $p(\mathbf{f}\mid\mathbf{y})$ is given. By using this property, we define a space of GP posteriors and propose GP-PCA.

\section{PCA for Gaussian processes (GP-PCA)}
\begin{figure}
    \centering
    \includegraphics[width=15cm]{fig/concept.pdf} \\
    \caption{Concept of GP-ePCA. An e-flat subspace $\mathcal{M}^\ast_e \subset \mathcal{S}^\ast$ is shown to be identical to an e-flat subspace $\mathcal{M}_e \subset \mathcal{S}$ to $\mathcal{T}^\ast$ through linear map $\mathcal{L}$.}
    \label{concept_of_GP-PCA}
\end{figure}

Similar to e-PCA and m-PCA, we consider two types of GP-PCA: GP-ePCA and GP-mPCA. In this study, we only explain the GP-ePCA, but the same argument holds for GP-mPCA. Let $p(f \mid \mathbf{y}_i)$ be a GP posterior obtained $\{X_i,\mathbf{y}_i\}$. When a set of posteriors $\mathcal{P} = \{p(f \mid \mathbf{y}_i)\}^I_{i=1}$ is given, the task of GP-ePCA is to estimate an e-flat subspace minimizing KL divergence between GP posteriors and their corresponding points on the subspace. However, it is nontrivial to define a structure of GPs since GP has an infinite-dimensional parameter.

This study shows that a set of GP posteriors is a finite-dimensional dually flat space under the assumption that each posterior has the same prior and reduces the task of GP-ePCA to a task of estimating a subspace on finite space. To explain our approach, we introduce the two probabilistic spaces shown in Fig~\ref{concept_of_GP-PCA}. One is a space consisting of Gaussian distributions for an output vector $\mathbf{f}:=f(X)$ corresponding to a training set $X=\bigcup^I_{i=1}X_i$. The other is a space consisting of Gaussian distributions for an output vector $\mathbf{f}_\ast:=f(X_\ast)$ corresponding to a set $X_\ast$ which is a union of the training input set and an arbitrary test input set $X_+$. The former is denoted by $\mathcal{S}$ and the latter is denoted by $\mathcal{S}^\ast$. Both $\mathcal{S}$ and $\mathcal{S}^\ast$ are dually flat spaces since they are a set of Gaussian distributions. Note that $\mathcal{S}^\ast$ can be regarded as an infinite-dimensional space since the cardinal of $X_+$ can be any number. In our approach, we define a space consisting of GP posteriors as a subspace on $\mathcal{S}^\ast$ denoted by $\mathcal{T}^\ast$. Then, we estimate an e-flat subspace $\mathcal{M}_e$ on $\mathcal{S}$ and transform $\mathcal{M}_e$ to $\mathcal{S}^\ast$ using an affine map $\mathcal{L} : \mathcal{S} \rightarrow \mathcal{T}^\ast$ instead of estimating an e-flat subspace $\mathcal{M}^\ast_e$ on $\mathcal{S}^\ast$. Since there is no guarantee that $\mathcal{L}(\mathcal{M}_e)$ is equivalent to $\mathcal{M}^\ast_e$, this study proves this.

In this Section, after defining $\mathcal{T}^\ast$ and GP-ePCA, we prove that $\mathcal{M}^\ast_e$ and $\mathcal{L}(\mathcal{M}_e)$ are equivalent. Next, we describe the standard algorithm and its sparse approximation algorithm.

\subsection{Definition of the structure of GP posteriors and GP-ePCA}

Let $X$ and $X_+$ be a union set of $\{X_i\}^I_{i=1}$ and test set. We consider estimating $p(\mathbf{f} \mid \mathbf{y}_i)$ given $\{X_i,\mathbf{y}_i\}$ in each task. Then, $i$-th task's predictive distribution for $X_+$ is derived as $q(\mathbf{f}_+\mid\mathbf{y}_i)=\int p(\mathbf{f}_+\mid\mathbf{f})p(\mathbf{f} \mid \mathbf{y}_i)d\mathbf{f}$. Suppose that GP posteriors have a common prior, $q(\mathbf{f}_+\mid\mathbf{y}_i)$ is determined uniquely given $p(\mathbf{f} \mid \mathbf{y}_i)$. From this fact, the affine subspace spanned by GP posteriors is defined as follows:
\begin{definition}
Let $X$, $X_+$ and $X_\ast$ be an input set, test set, and a union set of input and test sets, respectively. We denote the size of $X$ by $N$. Let $p(\mathbf{f}\mid\boldsymbol{\rho})$ be a Gaussian distribution with $\boldsymbol{\rho}$, where $\boldsymbol{\rho}$ is a pair of $N$-dimensional vector $\boldsymbol{\mu}$ and $N \times N$ positive-definite symmetric matrix $\mathbf{\Sigma}$. Then, a probability space consisting of GP posteriors corresponding to $f(X_\ast)$ with a common prior is defined by the following equation:
\begin{equation}
    \mathcal{T}^\ast=\Set{q(\mathbf{f}_+,\mathbf{f}\mid\boldsymbol{\rho})|q(\mathbf{f}_+,\mathbf{f}\mid\boldsymbol{\rho}) = p(\mathbf{f}_+\mid\mathbf{f})p(\mathbf{f}\mid\boldsymbol{\rho}), \forall \boldsymbol{\rho}},
    \label{eq_GP_posterior_space}
\end{equation}
where $p(\mathbf{f}_+\mid\mathbf{f})$ is a conditional distribution of the prior and $p(\mathbf{f} \mid \boldsymbol{\rho})$ is any Gaussian distribution with a parameter $\boldsymbol{\rho}$. In particular, when $X=X_\ast$ holds, i.e., $X_+$ is an empty set, $\mathcal{S}^\ast$ and $\mathcal{T}^\ast$ are denoted by $\mathcal{S}$ and $\mathcal{T}$, respectively.
\label{model_manifold_of_GP}
\end{definition}

Satisfying the assumption, $p(\mathbf{f}_+,\mathbf{f}\mid \mathbf{y}_i)$ is contained in $\mathcal{T}^\ast$. Let $\boldsymbol{\rho}_i=\{\boldsymbol{\mu}_i,\mathbf{\Sigma}_i\}$ be a parameter of $p(\mathbf{f}_+,\mathbf{f}\mid \mathbf{y}_i)$. $\boldsymbol{\rho}_i$ can be described as follows.
\begin{align}
    \boldsymbol{\mu}_i &= \boldsymbol{\mu}_0 + \mathbf{K}_i(\mathbf{K}_{ii}+\beta^{-1}\mathbf{I})^{-1}(\mathbf{y}_i-\boldsymbol{\mu}_{0i}), \label{eq_rho_mean} \\
    \mathbf{\Sigma}_i &= \mathbf{K} - \mathbf{K}_i(\mathbf{K}_{ii}+\beta^{-1}\mathbf{I})^{-1} \mathbf{K}^{\rm T}_i, \label{eq_rho_cov}
\end{align}
where $\mathbf{K}=k(X,X)$, $\mathbf{K}_i=k(X,X_i)$, $\mathbf{K}_{ii}=k(X_i,X_i)$, $\boldsymbol{\mu}_0=\mu_0(X)$ and $\boldsymbol{\mu}_{0i}=\mu_0(X_i)$. Therefore, we can define a space of GP posteriors as $\mathcal{T}^\ast$.

Since $\mathcal{S}^\ast$ is a dually flat space, $p(\mathbf{f}_\ast) \in \mathcal{S}^\ast$ can be represented by e-coordinate and m-coordinate denoted by $\boldsymbol{\xi}^\ast$ and $\boldsymbol{\zeta}^\ast$, respectively. We denote e-coordinate and m-coordinate for a point on $\mathcal{T}^\ast$ parameterized by $\boldsymbol{\rho}$ as $\boldsymbol{\xi}^\ast(\boldsymbol{\rho})$ and $\boldsymbol{\zeta}^\ast(\boldsymbol{\rho})$, respectively. From the definition of $\mathcal{T}^\ast$, when $X_\ast=X$, $\mathcal{S}=\mathcal{T}$ holds since $\boldsymbol{\mu}_\ast=\boldsymbol{\mu}$ and $\mathbf{\Sigma}_\ast=\mathbf{\Sigma}$ hold. It means that $\mathcal{T}(=\mathcal{S})$ is also a dually flat space. Therefore, we denote e-coordinate and m-coordinate of $p(\mathbf{f} \mid \boldsymbol{\rho}) \in \mathcal{T}$ by $\boldsymbol{\xi}=\boldsymbol{\xi}(\boldsymbol{\rho})$ and $\boldsymbol{\zeta}=\boldsymbol{\zeta}(\boldsymbol{\rho})$, respectively.

By using the definition of $\mathcal{T}^\ast$, we define GP-ePCA in the respective spaces of $\mathcal{S}^\ast$ and $\mathcal{S}$.
\begin{definition}
Let $\{\boldsymbol{\xi}^\ast(\boldsymbol{\rho}_i)\}^I_{i=1}$ be a set of GPs on the $\mathcal{T}^\ast$. Then, the objective function of GP-ePCA on $\mathcal{S}^\ast$ is defined as follows:
\begin{align}
    \hat{\mathbf{W}}^\ast,\hat{\mathbf{U}}^\ast &= \mathop{\rm arg~min}\limits_{\mathbf{W}^\ast,\mathbf{U}^\ast}E^\ast(\mathbf{W}^\ast,\mathbf{U}^\ast) \nonumber \\
    &= \mathop{\rm arg~min}\limits_{\mathbf{W}^\ast,\mathbf{U}^\ast}\sum^I_{i=1}\KL{\boldsymbol{\xi}^\ast(\boldsymbol{\rho}_i)}{\boldsymbol{\xi}^\ast(\mathbf{w}^\ast_i,\mathbf{U}^\ast)}.
    \label{eq_obj_func_GP_PCA_ast}
\end{align}
GP-ePCA estimating e-flat submanifold $\mathcal{M}^\ast_e\subset \mathcal{S}^\ast$ minimizing Eq.~\eqref{eq_obj_func_GP_PCA_ast} is called GP-ePCA$(\mathcal{S}^\ast)$. Here, $\boldsymbol{\xi}^\ast(\mathbf{w}^\ast_i,\mathbf{U}^\ast)$ is e-coordinate of $\mathcal{M}^\ast_e$ denoted by a linear combination of $\mathbf{U}^\ast=(\mathbf{u}^\ast_0,\mathbf{u}^\ast_1,\mathbf{u}^\ast_2,\ldots,\mathbf{u}^\ast_L)^{\rm T}$ with weight $(1,{\mathbf{w}^\ast_i}^{\rm T})$, where $\mathbf{w}^\ast_i:=(w^\ast_1,w^\ast_2,\ldots,w^\ast_L)^{\rm T}$.

Similarly, when $\{\boldsymbol{\xi}(\boldsymbol{\rho}_i)\}^I_{i=1}$ is observed, we call the ePCA minimizing the following equation GP-ePCA$(\mathcal{S})$.
\begin{align}
    \hat{\mathbf{W}},\hat{\mathbf{U}} &= \mathop{\rm arg~min}\limits_{\mathbf{W},\mathbf{U}}E(\mathbf{W},\mathbf{U}) \nonumber \\
    &= \mathop{\rm arg~min}\limits_{\mathbf{W},\mathbf{U}}\sum^I_{i=1}\KL{\boldsymbol{\xi}_i}{\boldsymbol{\xi}(\mathbf{w}_i,\mathbf{U})}.
    \label{eq_obj_func_GP_PCA}
\end{align}
Here, $\boldsymbol{\xi}_i$ and $\boldsymbol{\xi}(\mathbf{w}_i,\mathbf{U})$ are e-coordinate of $\mathcal{S}$ and $\mathcal{M}_e$, which is a linear combination of $\mathbf{U}=(\mathbf{u}_0,\mathbf{u}_1,\mathbf{u}_2,\ldots,\mathbf{u}_L)^{\rm T}$ with weight $(1,\mathbf{w}^{\rm T}_i)$, where $\mathbf{w}_i:=(w_1,w_2,\ldots,w_L)^{\rm T}$.
\label{definition_GP-PCA}
\end{definition}

In this study, we guarantee that GP-ePCA($\mathcal{S}^\ast$) is equivalent to GP-ePCA($\mathcal{S}$) by the following theorem.

\begin{theorem}
Let $\mathcal{M}^\ast_e$ and $\mathcal{M}_e$ be an e-flat subspace on $\mathcal{S}^\ast$ minimizing Eq.~\eqref{eq_obj_func_GP_PCA_ast} and an e-flat subspace on $\mathcal{S}$ minimizing Eq.~\eqref{eq_obj_func_GP_PCA}, respectively. Then, there is an affine map $\mathcal{L} : \mathcal{S} \rightarrow \mathcal{T}^\ast$ satisfying the following equation:
\begin{equation}
    \mathcal{M}^\ast_e = \mathcal{L}(\mathcal{M}_e).
    \label{eq_main_theorem}
\end{equation}
We prove the theorem below.

\label{main_theorem}
\end{theorem}

\subsection{Proof of Theorem~\ref{main_theorem}}
The proof of the Theorem~\ref{main_theorem} is composed of the proof of the following three statements.
\begin{enumerate}
    \renewcommand{\theenumi}{S\arabic{enumi}}
    \renewcommand{\labelenumi}{(\theenumi)}
    \item For $\forall \boldsymbol{\rho}$, there is $\mathcal{L}$ satisfying $\boldsymbol{\xi}^\ast(\boldsymbol{\rho}) = \mathcal{L}(\boldsymbol{\xi}(\boldsymbol{\rho}))$. \label{Statement1}
    \item For $\forall \boldsymbol{\rho}$, $\boldsymbol{\rho}'$, $\KL{\boldsymbol{\xi}^\ast(\boldsymbol{\rho})}{\boldsymbol{\xi}^\ast(\boldsymbol{\rho}')}=\KL{\boldsymbol{\xi}(\boldsymbol{\rho})}{\boldsymbol{\xi}(\boldsymbol{\rho}')}$ holds. \label{Statement2}
    \item For a subspace $\mathcal{M}^\ast_e \subset \mathcal{S}^\ast$ minimizing $E^\ast(\mathbf{W}^\ast,\mathbf{U}^\ast)$, $\mathcal{M}^\ast_e\subset\mathcal{T}^\ast$ holds. \label{Statement3}
\end{enumerate}

From (\ref{Statement1}) and (\ref{Statement2}), denoting a subspace minimizing Eq.~\eqref{eq_obj_func_GP_PCA} by $\mathcal{M}_e$, we can prove that $\mathcal{L}(\mathcal{M}_e)$ also minimizes Eq.~\eqref{eq_obj_func_GP_PCA_ast} in a set of subspaces on $\mathcal{T}^\ast$. However, since a subspace minimizing Eq.~\eqref{eq_obj_func_GP_PCA_ast} does not always lie on $\mathcal{T}^\ast$, we confirm this by (\ref{Statement3}).

To prove the statements, we present the following Lemmas.
\begin{lemma}
Let $\boldsymbol{\rho}$ be a parameter of $\mathcal{T}^\ast$. Then, there is an affine map $\mathcal{L}:\mathcal{S} \rightarrow \mathcal{T}^\ast$ satisfying the following equation.
\begin{equation}
    \boldsymbol{\xi}_\ast(\boldsymbol{\rho}) = \mathcal{L}(\boldsymbol{\xi}(\boldsymbol{\rho}))
\end{equation}
\label{lemma_e_flatness}
\end{lemma}
\begin{proof}
The proof is shown by Appendix~\ref{sec_appendix_proof_lemmas}
\end{proof}

\begin{lemma}
Let $\boldsymbol{\rho}$ and $\boldsymbol{\rho}'$ are two arbitrary parameters, and let us take two points $q(\mathbf{f}_\ast\mid \boldsymbol{\rho})$ and $q(\mathbf{f}_\ast\mid \boldsymbol{\rho}')$ in $\mathcal{T}^\ast$, and $q(\mathbf{f}\mid \boldsymbol{\rho})$ and $q(\mathbf{f}\mid \boldsymbol{\rho}')$ in $\mathcal{T}$. Then, the following equation holds:
\begin{equation}
    \KL{q(\mathbf{f}_\ast\mid \boldsymbol{\rho})}{q(\mathbf{f}_\ast\mid \boldsymbol{\rho}')}=\KL{q(\mathbf{f}\mid \boldsymbol{\rho})}{q(\mathbf{f}\mid \boldsymbol{\rho}')}
\end{equation}
\label{lemma_equivalent_of_kl_divergence}
\end{lemma}
\begin{proof}
The proof is shown by Appendix~\ref{sec_appendix_proof_lemmas}
\end{proof}

\begin{lemma}
Suppose $\mathcal{S}^\ast$ be a dually flat manifold and $\mathcal{T}^\ast \subset \mathcal{S}^\ast$ be a $K$-dimensional submanifold.
If $\mathcal{T}^\ast$ is a dually flat and a set of points $P = \{p(\mathbf{f}^\ast\mid \boldsymbol{\rho}_1),\ldots,p(\mathbf{f}^\ast\mid \boldsymbol{\rho}_I)\}\in \mathcal{T}^\ast$,
the $L$-dimensional e-flat submanifold $\mathcal{M}^\ast_e$ minimizing Eq.~\eqref{eq_obj_func_GP_PCA_ast} for $P$ is included in $\mathcal{T}^\ast$ when $L \le K$.
\label{lemma_equivalent_gppca}
\end{lemma}
\begin{proof}
The proof is shown by Appendix~\ref{sec_appendix_proof_lemmas}
\end{proof}

\begin{lemma}
Let $\boldsymbol{\rho}$ be a parameter of $\mathcal{T}^\ast$. Then, there is a linear mapping $\mathcal{L}:\mathcal{T} \rightarrow \mathcal{T}^\ast$ satisfying the following equation.
\begin{equation}
    \boldsymbol{\zeta}_\ast(\boldsymbol{\rho}) = \mathcal{L}(\boldsymbol{\zeta}(\boldsymbol{\rho}))
\end{equation}
\label{lemma_m_flatness}
\end{lemma}
\begin{proof}
The proof is shown by Appendix~\ref{sec_appendix_proof_lemmas}
\end{proof}

The proofs of (\ref{Statement1}) and (\ref{Statement2}) are obvious from Lemma~\ref{lemma_e_flatness} and Lemma~\ref{lemma_equivalent_of_kl_divergence}. From Lemma~\ref{lemma_equivalent_gppca}, (\ref{Statement3}) can be proved by showing that $\mathcal{T}^\ast$ is a dually flat for arbitrary test set $X_+$. When $X=X_\ast$, i.e., the test set is empty, then $\mathcal{T}$ is a dually flat since $\mathcal{T}=\mathcal{S}$. When $X\subset X_\ast$, by the linear relation proved in Lemma~\ref{lemma_e_flatness} and Lemma~\ref{lemma_m_flatness}, the Lemma also holds in the general case. Thus, Theorem~\ref{main_theorem} is proved.

\subsection{Algorithm of GP-ePCA}
From the above  discussion, GP-ePCA$(\mathcal{S}^\ast)$ can be reduced to GP-ePCA$(\mathcal{S})$. In this Section, we explain a concrete algorithm of GP-ePCA$(\mathcal{S})$.
\subsubsection{GP-ePCA}
Let $X_i \in \mathcal{X}^{N_i}$ and $\mathbf{y}_i \in \mathbb{R}^{N_i}$ be a training input and corresponding output dataset of $i$-th task, where $N_i$ is the size of $X_i$. We denote a union set of the input sets by $X$, i.e., $X=\bigcup^I_{i=1}X_i$ and define the probability space of GP posteriors as Eq.~\eqref{eq_GP_posterior_space}. Then, we denote the GP posterior given $(X_i,\mathbf{y}_i)$ by $q(f\mid \boldsymbol{\rho}_i)$. From Theorem~\ref{main_theorem}, the task of GP-ePCA$(\mathcal{S})$ is to estimate a subspace $\mathcal{M}_e$ for $\{p(\mathbf{f}\mid \boldsymbol{\rho}_i)\}^I_{i=1}$ and transform $\mathcal{M}_e$ to $\mathcal{T}^\ast$.

In training phase, GP-ePCA calculates the $\{\boldsymbol{\rho}_i\}^I_{i=1}$ and transforms the m-coordinates $\mathbf{Z} := (\boldsymbol{\zeta}_1,\boldsymbol{\zeta}_2,\cdots,\boldsymbol{\zeta}_I)$. The $\boldsymbol{\rho}_i$ is calculated using Eqs.~\eqref{eq_rho_mean} and \eqref{eq_rho_cov} and from $\boldsymbol{\zeta}_i=(\boldsymbol{\eta}_i,{\rm vec}(\mathbf{H}_i))$ is transformed from $\boldsymbol{\rho}_i$ by $\boldsymbol{\eta}_i=\boldsymbol{\mu}_i$ and $\mathbf{H}_i=\mathbf{\Sigma}_i+\boldsymbol{\mu}_i\boldsymbol{\mu}_i^{\rm T}$. Next, GP-ePCA$(\mathcal{S})$ estimates the subspace $\mathcal{M}_e$ using e-PCA. That is, estimating ${\hat \mathbf{W}}$ and ${\hat \mathbf{U}}$ minimizing Eq.~\eqref{eq_obj_func_GP_PCA} through gradient descent iterations. Algorithm~\ref{alg_standard_GP_ePCA} shows the summary of the algorithm.

In the prediction phase, GP-ePCA predict outputs corresponding to a test data $x$ using the following equations.
\begin{align*}
    \mu_i(x) =& \boldsymbol{\mu}_0(x)+\mathbf{k}^{\rm T}(x)\mathbf{K}^{-1}\left(\hat{\mathbf{\Theta}}^{-1}_i\hat{\boldsymbol{\theta}}_i-\boldsymbol{\mu}_0\right) \\
    \sigma_i(x,x') =& k(x,x')+\mathbf{k}^{\rm T}(x)\mathbf{K}^{-1}\left(-\frac{1}{2}\hat{\mathbf{\Theta}}^{-1}_i-\mathbf{K}\right)\mathbf{K}^{-1}\mathbf{k}(x)
\end{align*}

Since this algorithm requires calculating the inverse matrix, the calculation cost of the algorithm becomes $\mathcal{O}(N^3)$, where $N:=\sum^I_{i=1}N_i$. Since this algorithm is impractical, we derive a faster approximation below.

\begin{algorithm}[H]
\caption{GP-ePCA}
\label{alg_standard_GP_ePCA}
\begin{algorithmic}
    \State Given $\{(X_i,\mathbf{y}_i)\}^I_{i=1}$, kernel $k$ and $\beta$.
    \State Initialize $\mathbf{U}$ and $\mathbf{W}$
    \For{$i = 1 \, \ldots \, I$}
        \State $\boldsymbol{\mu}_i \leftarrow \boldsymbol{\mu}_0 + \mathbf{K}_i(\mathbf{K}_{ii}+\beta^{-1}\mathbf{I})^{-1}(\mathbf{y}_i+\boldsymbol{\mu}_{i0})$
        \State $\mathbf{\Sigma}_i \leftarrow \mathbf{K} - \mathbf{K}_i(\mathbf{K}_{ii}+\beta^{-1}\mathbf{I})^{-1} \mathbf{K}^{\rm T}_i$
        \State $\boldsymbol{\zeta}_i \leftarrow (\boldsymbol{\mu}_i,{\rm vec}(\mathbf{\Sigma}_i+\boldsymbol{\mu}_i\boldsymbol{\mu}_i^{\rm T}))$
    \EndFor

    \While{stopping criterion is met}
        \State $\mathbf{W}^{\rm (new)} \leftarrow \mathbf{W}^{\rm (old)}-\varepsilon(\hat{\mathbf{Z}}-\mathbf{Z})^{\rm T}\mathbf{U}$
        \State $\mathbf{U}^{\rm (new)} \leftarrow \mathbf{U}^{\rm (old)}-\varepsilon\mathbf{W}(\hat{\mathbf{Z}}-\mathbf{Z})$
        \For{$i = 1 \, \ldots \, I$}
            \State $\hat{\boldsymbol{\xi}}_i \leftarrow \hat{\mathbf{w}}^{\rm T}_i\hat{\mathbf{U}}$
        \EndFor
    \EndWhile
\end{algorithmic}
\end{algorithm}

\subsubsection{Sparse GP-ePCA}
Most sparse approximation methods for GP reduce a calculation cost by approximating the gram matrix for input set using inducing points~\cite{Liu2020}. Let $X_m$ and $\mathbf{K}$ be a set of inducing points and gram matrix between inputs. The gram matrix is approximated as
\begin{equation*}
    \mathbf{K} \approx \mathbf{K}_m\mathbf{K}^{-1}_{mm}\mathbf{K}^{\rm T}_m,
\end{equation*}
where $\mathbf{K}_m=k(X,X_m)$, $\mathbf{K}_{mm}=k(X_m,X_m)$. By using this approximation, we consider a set of GPs for $\mathbf{f}_m:=f(X_m)$ instead of a set of GPs for $f(X)$. Denoting the set of GPs for $\mathbf{f}_m$ by $\mathcal{S}_m$, the sparse GP-ePCA estimates a subspace on $\mathcal{S}_m$ and transforms the subspace to $\mathcal{T}^\ast$. Then, we reduce the calculation cost of GP-ePCA from $\mathcal{O}(N^3)$ to $\mathcal{O}(m^3)$, where $m$ is the size of inducing points.

We adopt a sparse GP based on variational inference proposed by Titsias~\cite{Titsias2009}. The variational inference-based sparse GP minimizes the KL-divergence between a true posterior $p(\mathbf{f},\mathbf{f}_m \mid \mathbf{y})$ and variational distribution $q(\mathbf{f}, \mathbf{f}_m)$, that is,
\begin{equation*}
    \KL{q(\mathbf{f}, \mathbf{f}_m)}{p(\mathbf{f},\mathbf{f}_m \mid \mathbf{y})}=\int q(\mathbf{f}, \mathbf{f}_m) \ln \frac{q(\mathbf{f}, \mathbf{f}_m)}{p(\mathbf{f},\mathbf{f}_m \mid \mathbf{y})}d\mathbf{f} d\mathbf{f}_m.
\end{equation*}
Then, the variational distribution minimizing the equation is derived as follows:
\begin{align*}
    q(\mathbf{f}_m) =& \mathcal{N}(\mathbf{f}_m \mid \boldsymbol{\mu},\mathbf{\Sigma}) \\
    \boldsymbol{\mu} =& \boldsymbol{\mu}_{m0} + \mathbf{K}_{mm}\mathbf{A}^{-1}_{mm}\mathbf{K}^{\rm T}_{m}(\mathbf{y}-\boldsymbol{\mu}_0) \\
    \mathbf{\Sigma} =& \beta^{-1}\mathbf{K}_{mm}\mathbf{A}^{-1}_{mm}\mathbf{K}_{mm},
\end{align*}
where $\mathbf{A}_{mm}=\beta^{-1}\mathbf{K}_{mm}+\mathbf{K}_m\mathbf{K}^{\rm T}_m$. The predictive distribution for new input $x_+$ is as follows:
\begin{align*}
    q(f(x_+)) =& \mathcal{N}(f(x_+) \mid \mu(x_+),\sigma(x_+,x_+)) \\
    \mu(x_+) =& \mu_{0}(x_+) + \mathbf{k}^{\rm T}_m(x_+)\mathbf{K}^{-1}_{mm}\boldsymbol{\mu} \\
    \sigma(x_+,x_+) =& \beta^{-1}\mathbf{k}^{\rm T}_m(x_+)\mathbf{K}^{-1}_{mm}\mathbf{\Sigma}\mathbf{K}^{-1}_{mm}\mathbf{k}_m(x_+),
\end{align*}

We regard $\boldsymbol{\mu}$ and $\mathbf{\Sigma}$ as a parameter of Eq.~\eqref{eq_GP_posterior_space}. That is, denoting a parameter of $i$-th task's variational distribution by $\boldsymbol{\rho}_i=\{\boldsymbol{\mu}_i,\mathbf{\Sigma}_i\}$, the sparse GP-ePCA estimates a subspace minimizing Eq.~\eqref{eq_obj_func_GP_PCA} for $\{\boldsymbol{\rho}_i\}^I_{i=1}$ and transforms the subspace to $\mathcal{T}^\ast$ by the affine map $\mathcal{L}$.

In practice, to stabilize the sparse GP-ePCA, we re-parametrize $\boldsymbol{\rho}_i$ as follows:
\begin{align*}
    \boldsymbol{\mu}' =& \mathbf{K}^{-1}_{mm}\boldsymbol{\mu}, \\
    \mathbf{\Sigma}' =& \mathbf{K}^{-1}_{mm}\mathbf{\Sigma}\mathbf{K}^{-1}_{mm}.
\end{align*}
We denote a space of $\boldsymbol{\rho}'=\{\boldsymbol{\mu}',\mathbf{\Sigma}'\}$ by $\mathcal{S}'_m$.
Letting $\boldsymbol{\theta}'={\mathbf{\Sigma}'}^{-1}\boldsymbol{\mu}'$, $\mathbf{\Theta}' = -\frac{1}{2}{\mathbf{\Sigma}'}^{-1}$, $\boldsymbol{\theta}=\mathbf{\Sigma}^{-1}\boldsymbol{\mu}$ and $\mathbf{\Theta} = -\frac{1}{2}\mathbf{\Sigma}^{-1}$, the following relationships between $\boldsymbol{\xi}'(\boldsymbol{\rho})=(\boldsymbol{\theta}',{\rm vec}(\mathbf{\Theta}'))$ and $\boldsymbol{\xi}(\boldsymbol{\rho})=(\boldsymbol{\theta},{\rm vec}(\mathbf{\Theta}))$ hold.
\begin{align*}
    \mathbf{\Theta} =& \mathbf{K}^{-1}_{mm}\mathbf{\Theta}'\mathbf{K}^{-1}_{mm}. \\
    \boldsymbol{\theta} =& \mathbf{K}^{-1}_{mm}\mathbf{\Theta}'\mathbf{K}^{-1}_{mm}\boldsymbol{\mu}_0+\mathbf{K}^{-1}_{mm}\boldsymbol{\theta}'.
\end{align*}
Furthermore, using the equations, we can show the equivalence between the KL-divergence of $\boldsymbol{\xi}'$ and that of $\boldsymbol{\xi}$. That is, for any $\boldsymbol{\rho}_i$ and $\boldsymbol{\rho}_j$, the following equation holds:
\begin{equation*}
    \KL{\boldsymbol{\xi}'(\boldsymbol{\rho}_i)}{\boldsymbol{\xi}'(\boldsymbol{\rho}_j)}=\KL{\boldsymbol{\xi}(\boldsymbol{\rho}_i)}{\boldsymbol{\xi}(\boldsymbol{\rho}_j)}
\end{equation*}
From the above relationships, $\mathcal{S}_m$ and $\mathcal{S}'_m$ are isomorphic. Therefore, we estimate a subspace on $\mathcal{S}'_m$ instead of estimating a subspace on $\mathcal{S}_m$. The algorithm is summarized by algorithm~\ref{alg_sparse_approx_GP_ePCA}.

\begin{algorithm}[H]
\caption{Sparse GP-ePCA}
\label{alg_sparse_approx_GP_ePCA}
\begin{algorithmic}
    \State Given $\{(X_i,\mathbf{y}_i)\}^I_{i=1}$, kernel $k$, $\beta$ and $X_m$.
    \State Initialize $\mathbf{U}$ and $\mathbf{W}$
    \For{$i = 1 \, \ldots \, I$}
        \State $\mathbf{A}_{mm} \leftarrow \beta^{-1}\mathbf{K}_{mm}+\mathbf{K}_m\mathbf{K}^{\rm T}_m$
        \State $\boldsymbol{\mu}_i \leftarrow \mathbf{A}^{-1}_{mm}\mathbf{K}^{\rm T}_{m}(\mathbf{y}-\boldsymbol{\mu}_0)$
        \State $\mathbf{\Sigma}_i \leftarrow \beta^{-1}\mathbf{A}^{-1}_{mm}$
        \State $\boldsymbol{\zeta}_i \leftarrow (\boldsymbol{\mu}_i,{\rm vec}(\mathbf{\Sigma}_i+\boldsymbol{\mu}_i\boldsymbol{\mu}_i^{\rm T}))$
    \EndFor
    \While{stopping criterion is met}
        \State $\mathbf{W}^{\rm (new)} \leftarrow \mathbf{W}^{\rm (old)}-\varepsilon(\hat{\mathbf{Z}}-\mathbf{Z})^{\rm T}\mathbf{U}$
        \State $\mathbf{U}^{\rm (new)} \leftarrow \mathbf{U}^{\rm (old)}-\varepsilon\mathbf{W}(\hat{\mathbf{Z}}-\mathbf{Z})$
        \For{$i = 1 \, \ldots \, I$}
            \State $\hat{\boldsymbol{\xi}}_i \leftarrow \hat{\mathbf{w}}^{\rm T}_i\hat{\mathbf{U}}$
        \EndFor
    \EndWhile
\end{algorithmic}
\end{algorithm}

\section{Related works}
\subsection{Meta-learning and multi-task learning}
Meta-learning is a framework that estimates a common knowledge of tasks through similar but different learning tasks and adapts to new tasks~\cite{Vilalta2002,Hospedales2020,Huisman2021}. As a framework similar to meta-learning, multi-task learning improves the predictive accuracy of each task by estimating a common knowledge of tasks~\cite{Zhang2017}. Since the approach of the meta-learning for GP is the same as that of the multi-task learning for GP, we explain the conventional meta-learning and multi-task learning methods.

Most conventional meta-learning methods for GP estimate a prior of each task. The simplest approach is to estimate a common prior between tasks, and the prior is estimated based on hierarchical Bayes modeling or deep neural network (DNN)~\cite{Fortuin2020,Schwaighofer2005, Yu2005,Rothfuss2021}. The approach models common knowledge between tasks but does not model individual knowledge of each task. As an approach for estimating common and individual knowledge of the tasks, there are feature learning and cross-covariance approaches. In the feature learning approach, meta-learning selects input features in each task by estimating hyperparameters of auto-relevant determination kernel or multi-kernel~\cite{Srijith2014,Titsias2011}. In the cross-covariance approach, meta-learning assumes that a covariance function of priors is defined by the Kronecker product of a covariance function of samples and that of tasks and estimates the covariance function of tasks~\cite{Bonilla2007}. In geostatistics, the approach is called linear models of coregionalization (LMC) and various methods have been proposed~\cite{Alvarez2011}. The conbination of feature learning and cross-covariance approaches has been proposed~\cite{Li2018}. Although these approaches estimate common and individual knowledge of the task, they estimate covariance function but do not estimate the mean function. GP-PCA estimates a subspace on a space of GP posteriors. Therefore, GP-PCA enables the estimation of mean and covariance functions of each task, including a new task.

This study interprets meta-learning for GP from the information geometry viewpoint. Transfer learning and meta-learning are often addressed from the information geometry perspective~\cite{Takano2016,Waytowich2016,Flennerhag2019}. However, to our best knowledge, there is no research of meta-learning for GP addressed from the information geometry viewpoint.

\subsection{Dimension reduction methods for probabilistic distributions}
Dimensionality reduction techniques for probability distributions have been proposed in various fields. For example, there are dimension reduction techniques of a set of categorical distributions~\cite{Hofmann2001} and a set of mixture models~\cite{Akaho2008,Covarrubias2013}. Especially, e-PCA and m-PCA are closely related to this study~\cite{Collins2001,Akaho2004}. e-PCA and m-PCA are proposed in the context of information geometry for the dimension reduction method of a set of exponential distribution families, which becomes the basic framework for conducting this study. This study differs from previous studies in that it deals with GP sets that are infinite-dimensional stochastic processes.

\subsection{Functional PCA}
GP-PCA can also be interpreted as a functional PCA (fPCA). fPCA is a method for estimating eigenfunctions from a set of functions~\cite{Shang2014}. Let $\{f_i\}^I_{i=1}$ be a set of functions. fPCA estimates eigenfunctions to minimize the following objective function.
\begin{align*}
    F_{\rm fPCA}=&\sum^I_{i=1}\int (f_i(x)-h(x)-\bar{f}(x))^2p(x)dx, \\
    & s.t. \quad \int h^2(x)p(x)dx=1,
\end{align*}
where $\bar{f}=\frac{1}{I}\sum^I_{i=1}f_i(x)$. In fPCA, each function is represented as a linear combination of $M$ basis functions. Let $\mathbf{g}(x)=(g_1(x),g_2(x),\ldots,g_I(x))^{\rm T}$, $f_i$ is obtained as
\begin{align*}
    f_i(x) =& \sum^M_{m=1}r_{im}g_m(x) \\
    =& \mathbf{r}^{\rm T}_i\mathbf{g}(x),
\end{align*}
where $\mathbf{r}_i=(r_{i1},r_{i2},\ldots,r_{iM})^{\rm T}$. $h(x)$ is represented as a linear combination of $\mathbf{g}(x)$: $h(x)=\mathbf{s}^{\rm T}\mathbf{g}(x)$. By using the equations, the objective function of fPCA is rewritten as follows:
\begin{align*}
    F_{\rm fPCA}=&\sum^I_{i=1} (\mathbf{r}_i-\mathbf{s}-\bar{\mathbf{r}})^{\rm T}\mathbf{G}(\mathbf{r}_i-\mathbf{s}-\bar{\mathbf{r}}), \\
    & s.t. \quad \mathbf{s}^{\rm T}\mathbf{G}\mathbf{s}=1,
\end{align*}
where $\bar{\mathbf{r}}=1/I\sum^I_{i=1}\mathbf{r}_i$ and $\mathbf{G}=\int\mathbf{g}(x)\mathbf{g}^{\rm T}(x)p(x)dx$.

In practice, $f_i$ is estimated using linear regression from the dataset. That is, the fPCA algorithm consists of two processes: estimating $f_i$ from $\{X_i,\mathbf{y}_i\}$ and estimating $h$ from $\{f_i\}^I_{i=1}$. When $f_i$ is estimated as GP, GP-PCA is equivalent to fPCA. Therefore, GP-PCA can be interpreted as fPCA considering the estimated function and confidence of the function.

\begin{figure}
    \centering
    \begin{tabular}{ccc}
    \includegraphics[width=4.5cm]{fig/true_func_10_4.pdf}&
    \includegraphics[width=4.5cm]{fig/fitting_GP_10_4.pdf}&
    \includegraphics[width=4.5cm]{fig/fitting_GP_ePCA_10_4.pdf} \\
    (a) True functions& (b) GP& (c) GP-ePCA \\
    \end{tabular}
    \caption{True and mean functions of GP-ePCA and GP using $10$ samples / task for training. The scatter plot is the training data. The colors of the scatter plot and function indicate a value of $z_i$.}
    \label{fig_fitting}
\end{figure}

\section{Experimental results}
In this Section, we demonstrate the effectiveness of the proposed method as multi-task learning and meta-learning. We compare the performance of GP-ePCA and that of GP for training and test tasks.
\subsection{Artificial dataset}
In this experiment, we compare the methods using an artificial dataset. The artificial dataset is generated from the following equations.
\begin{align}
    y_{in} &= z_i \sin(2\pi x_{in}) + (1-z_i)((-x_{in}-1)^2+1) +\varepsilon_{in} \label{eq_output_generator} \\
    x_{in} &\sim U(0,1) \label{eq_input_generator}
\end{align}
where $U[a,b]$ means a continuous uniform distribution of interval $[a,b]$, $\mathbf{z}_i$ is a latent variable of $i$-th task, and $\varepsilon_{in}$ is a Gaussian noise with mean $0$ and variance $\beta^{-1}=0.2^2$.

To verify the performance of GP-ePCA for training and test tasks, training data and test data of training and test tasks are necessary. In training tasks, after $z_i$ is sampled $20$ points at even intervals from $0$ to $1$, we sample $N$ training data and $100$ test data of each task according to Eqs.~\eqref{eq_output_generator} and \eqref{eq_input_generator}. Figure~\ref{fig_fitting}(a) shows the sampled true functions of training tasks. In test tasks, after sampling latent variables of the test tasks at $100$ samples, we sample $N$ training data and $100$ test data for test tasks. Then, the training data of the test tasks are used to determine a point on an estimated subspace in GP-ePCA. In GP, the posteriors for the test tasks are estimated from the training data of the task. The performance of each method is evaluated using an average of root mean square error (RMSE) for test data in each task. We calculate the average and standard deviation when $T=5$ times iterates.

GP has hyperparameters, which are kernel and variance of observation noise. In this experiment, we use RBF kernel $k(x,x')=\exp(-(x-x')^2/2l^2)$. Then, the hyperparameters of GP are a length scale $l$ of the RBF kernel and variance of an observation noise $\beta^{-1}$. For a fair evaluation, these hyperparameters are set to an identical value manually in GP-ePCA and GP. In GP-ePCA, the dimension of the latent space is also a hyperparameter, and we set the size to $H=1$.

Figure~\ref{fig_fitting} shows representative results of GP-ePCA and GP. In a small sample size (Fig.~\ref{fig_fitting}(b)), GP cannot predict output in an area without training data since a mean function of GP is close to prior. On the other hand, GP-ePCA can predict output in an area without training data since mean functions are smoothed by transferring knowledge from other tasks (Fig.~\ref{fig_fitting}(c)). Figure~\ref{fig_gene_error_artificial} shows the RMSE of the methods for training and test tasks when $N$ is varied. From these results, we obtain that GP-ePCA performs better than GP in both training and test tasks.

\begin{figure}
    \centering
    \begin{tabular}{cc}
    \includegraphics[width=6cm]{fig/gene_error.pdf}&
    \includegraphics[width=6cm]{fig/gene_error_test.pdf} \\
    (c) Train task& (d) Test task \\
    \end{tabular}
    \caption{RMSE of the GP-ePCA and GP for artificial dataset. Line and bar means average and standard deviation of RMSE, respectively.}
    \label{fig_gene_error_artificial}
\end{figure}

\subsection{Van der Pol oscillator}
\begin{figure}
    \centering
    \begin{tabular}{cc}
    \includegraphics[width=6cm]{fig/latent_variable_5_3.pdf}&
    \includegraphics[width=6cm]{fig/drift_ode_5_3_0_1.pdf} \\
    (a) True parameter and estimated latent variable& (b) Train task with $\alpha=0.1$ \\
    \includegraphics[width=6cm]{fig/drift_ode_5_3_1.pdf}&
    \includegraphics[width=6cm]{fig/drift_ode_test_5_3_0_66.pdf} \\
    (c) Train task with $\alpha=1.0$& (d) Test task with $\alpha=0.66$ \\
    \end{tabular}
    \caption{Results for VDP, when $40$ sequence / task used for training. (a) True parameter and estimated parameter spaces by GP-ePCA. The markers outlined in black represent the training data. The colors mean a value of $\alpha$. (b) -- (d) True vector and estimated vector fields for each parameter.}
    \label{fig_ODE}
\end{figure}

In this experiment, we apply the proposed method to the modeling of the Van der Poll (VDP) oscillator as an example of multi-task learning and meta-learning. VDP is an ordinary differential equation (ODE) described by the following equation~\cite{Pol1926}.
\begin{equation}
    \frac{d^2x}{dt^2}-\alpha(1-x^2)\frac{dx}{dt}+x=0,
    \label{eq_vdp}
\end{equation}
where $\alpha\geq0$. After sampling VDP's parameters at $I$ samples, $J$ sequences are generated according to each VDP. Denoting $i$-th VDP's parameter by $\alpha_i$, let $X^i_n=\{x^i_{n1},x^i_{n2},\cdots,x^i_{nJ}\}$ and $T^i_n=\{t^i_{n1},t^i_{n2},\cdots,t^i_{nJ}\}$ be $n$-th sequence generated from VDP with $\alpha_i$ and the corresponding observation time points, respectively. Then, the task is to estimate each VDP's model $dx/dt=f_i(x)=f(x\mid \alpha_i)$ and the parameter space of $\alpha$ from $\{X^i_n\}^{I,N}_{i=1,n=1}$.

\begin{figure}
    \centering
    \begin{tabular}{cc}
    \includegraphics[width=6cm]{fig/gene_error_ODE.pdf}&
    \includegraphics[width=6cm]{fig/gene_error_ODE_test.pdf} \\
    (c) train task& (d) test task \\
    \end{tabular}
    \caption{RMSE of GP-ePCA and GP for VDP. The line and bar indicate the average and standard deviation of RMSE, respectively.}
    \label{fig_gene_error_ODE}
\end{figure}

In this experiment, considering $dx/dt$ as $v^i_j:=\frac{x^i_{j+1}-x^i_j}{t^i_{j+1}-t^i_j}$, we reduce each task to a regression problem to estimate $h_i$ from $\{(x^i_{nj},v^i_{nj})\}^{N,J}_{n=1,j=1}$. Since an input distribution of each task varies with the ODE's parameter, multi-task learning of ODEs causes a domain shift between tasks in general. Therefore, the task is more difficult than standard multi-task learning. To mitigate this problem, we consider a case that the initial points of the sequences are identical between tasks. We sampled $10$ VDP parameters at even intervals from $0.1$ to $1.0$. Then, $N$ sequences of each task are generated using Eq.~\eqref{eq_vdp}, where each sequence has data of $5$ times. We generated $100$ sequences randomly according to each VDP as test data to evaluate the RMSE.

Figure~\ref{fig_ODE} shows representative results of the estimated parameter space and ODEs. As shown in Fig.~\ref{fig_ODE}(a), GP-ePCA can estimate the parameter space of VDP. The ODE estimated by GP-ePCA tends to be closer to the true ODE than that of GP (Fig.~\ref{fig_ODE}(b)--(c)). We obtain that GP-ePCA performs better than GP (Fig.~\ref{fig_gene_error_ODE}). From these results, the proposed method can estimate the parameter space of the set of the differential equations.

\section{Conclusion}
In this study, we proposed a PCA for a set of GP posteriors. Since a structure of a set of GPs is nontrivial, we defined the space of GP posteriors and proved that space becomes a finite dually flat subspace. By using this fact, PCA for a set of GP posteriors can be regarded as an e-PCA or m-PCA for a set of finite-dimensional multivariate normal distributions. Furthermore, we proposed a fast algorithm, which reduces the calculation order from $\mathcal{O}(N^3)$ to $\mathcal{O}(m^3)$, where $N \gg m$. We demonstrated that the proposed algorithm can be applied to multi-task learning and meta-learning.

\subsection*{Acknowledgments}
This work was supported by JSPS KAKENHI grant numbers 17H01793 and 20K19865.

redpajama_set_name: RedPajamaArXiv


About UsOur mission is to share the Love of God with every man, woman, and child on the planet and to share the truth of the salvation found in Jesus Christ. We use the Internet and emerging technologies to give people multiple opportunities to know and accept Jesus Christ.
ThreeAngels.com is a ministry that is supported by members sharing Bible based truth. Videos and other information posted on ThreeAngels.com is shared on the site with all members and visitors. Many of the links to videos and other content that are posted to ThreeAngels.com are distributed all over the Internet. ThreeAngels.com provides a vehicle for sharing Bible based truth with millions of people all over the planet. The truth of God's Word will be made easily available to everyone seeking and not seeking it.
We also believe that sharing the love of God should be done in many forms: by word of mouth, with music, food, clothing, housing, providing education, giving medical aid, hugs, words of encouragement, and most importantly with prayer.
ThreeAngels.com supports Christian Education worldwide. Members from every nation and language are encouraged to participate in the spreading of the gospel message. We trust in the Holy Spirit to provide us with believers who are willing to participate in the work of kingdom building. We ask all believers in Jesus Christ to help in sharing the message of salvation wherever they are.
ThreeAngels.com © COPYRIGHT 2018. ALL RIGHTS RESERVED.


text:
Kind Lady
ThrillerDrama19351h 16m
#133686 Ranked Movie
Kind Lady (1935)
Because You d Kind Lady
Thriller Drama 19351h 16m
Mary Herries is a rich woman with a habit of contributing to those less fortunate. On her way home from a concert on Christmas Eve she discovers a poor, would-be artist outside her estate. Mary takes pity on this artist, Henry Abbott, and gives him some food and money. After taking him in, she finds herself somewhat attracted to this artist; he is handsome, and quite knowledgeable of fine art, especially the paintings in Mary's extensive collection. However, when she discovers that Henry has a wife and small child he struggles to support, she gives him some money and hand-me-downs and sends him on his way. A few days later he shows up with some of his own paintings (which are absolutely awful) as well as some items he stole from Mary's house on Christmas Eve. Henry demands a large amount of money for his paintings, which Mary eventually pays. She then discovers that Henry has left his wife and baby outside in the rain. His wife collapses and Mary, out of pity, lets Henry and his family stay with her until his wife is well. Soon, Mary's servants have all quit, tired of dealing with Henry and his family, who are unreasonable and greedy. Once the servants are gone, Henry's extended family arrives, and Mary discovers to her horror that Henry's "family" is actually a gang of art thieves, planning to imprison her in her own house to gain control of her art collection.Kind Lady featuring Aline MacMahon and Basil Rathbone is streaming via tv everywhere with TCM. It's a drama and thriller movie with a better than average IMDb audience rating of 6.8 (552 votes).
Where to Watch Kind Lady
Kind Lady is not available to watch online.
Get notified when you can stream it.
Bachelor Mother (1939)
Mark of the Vampire (1935)
Emmet Otter's Jug-Band Christmas (1977)
Mon Oncle Antoine (1971)
A Charlie Brown Christmas (1965)
Klaus (2019)
Bad Santa (2003)
It's a Very Merry Muppet Christmas Movie (2002)
Feast of the Seven Fishes (2019)
Frosty the Snowman (1969)
Home Alone (1990)
Evelyn (2002)
Where God Left His Shoes (2007)
Andy Hardy Meets Debutante (1940)
Trading Places (1983)
The Shop Around the Corner (1940)
A Bad Moms Christmas (2017)
Last Holiday (2006)
Dolly Parton's Christmas of Many Colors: Circle of Love (2016)
Jack Frost (1979)
In the Bleak Midwinter (1995)
My Favorite Season (1993)
A Detour for Christmas (2017)
Mother Krampus (2017)
A Christmas in New York (2017)
Happy Christmas (2014)
The Planet of Junior Brown (1997)
White Reindeer (2013)
Uncle Nick (2015)
A Christmas Star (2015)
March of the Wooden Soldiers (1934)
George B. Seitz
Mary Herries
Henry Abbott
Frank Albertson
Peter Santard
Dudley Digges
Mr. Edwards
Doris Lloyd
Lucy Weston
Nola Luxford
Rose, Mary's Maid
Murray Kinnell
Eily Malyon
Mrs. Edwards
Justine Chase
Ada Abbott
Barbara Shields
Aggie Edwards
Mr. Foster
Frank Reicher
Gustave Roubet (as Frank Reigher)
Jimmy Aubrey
Soapbox Orator (uncredited)
E. E. Clive
Grammaphone Man (uncredited)
Charles Coleman
Doorman (uncredited)
Elspeth Dudgeon
Lady Emily (uncredited)
Colin Kenny
Second Scotland Yard Man (uncredited)
Wilfred Lucas
First Scotland Yard Man (uncredited)
Milton A. Owen
Servant (uncredited)
Tempe Pigott
Flower Woman (uncredited)
Lucien Hubbard
Bernard Schubert
Edward Ward
Available to stream on a TV everywhere service (TCM).
Not available to rent or buy.
#133,686 Ranked Movie
#132 Ranked in Movies on TCM
#1316 Ranked in Christmas Movies
#13277 Ranked in Thriller Movies
Kind Lady has a better than average IMDb audience rating of 6.8 (552 votes). The movie is somewhat popular with Reelgood users lately.
About Kind Lady
Kind Lady Overview
Mary Herries is a rich woman with a habit of contributing to those less fortunate. On her way home from a concert on Christmas Eve she discovers a poor, would-be artist outside her estate. Mary takes pity on this artist, Henry Abbott, and gives him some food and money. After taking him in, she finds herself somewhat attracted to this artist; he is handsome, and quite knowledgeable of fine art, especially the paintings in Mary's extensive collection. However, when she discovers that Henry has a wife and small child he struggles to support, she gives him some money and hand-me-downs and sends him on his way. A few days later he shows up with some of his own paintings (which are absolutely awful) as well as some items he stole from Mary's house on Christmas Eve. Henry demands a large amount of money for his paintings, which Mary eventually pays. She then discovers that Henry has left his wife and baby outside in the rain. His wife collapses and Mary, out of pity, lets Henry and his family stay with her until his wife is well. Soon, Mary's servants have all quit, tired of dealing with Henry and his family, who are unreasonable and greedy. Once the servants are gone, Henry's extended family arrives, and Mary discovers to her horror that Henry's "family" is actually a gang of art thieves, planning to imprison her in her own house to gain control of her art collection.
Related Genres: ThrillerDrama
Related Tags: ChristmasHoliday
Country of origin: America
Available to watch on: TCM
Original Release Date: Dec 6, 1935


text:
Instructions below are relative to the .\ComputeAdmin folder of the [AzureStack-Tools repo](..).

```powershell
Import-Module .\AzureStack.ComputeAdmin.psm1
```

##Add a VM image to the Marketplace with PowerShell

1. Prepare a Windows or Linux operating system virtual hard disk image in VHD format (not VHDX).
    -   For Windows images, the article [Upload a Windows VM image to Azure for Resource Manager deployments](https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-upload-image/) contains image preparation instructions in the **Prepare the VHD for upload** section.
    -   For Linux images, follow the steps to
        prepare the image or use an existing Azure Stack Linux image as described in
        the article [Deploy Linux virtual machines on Azure
        Stack](https://azure.microsoft.com/en-us/documentation/articles/azure-stack-linux/).

2. Add the VM image by invoking the Add-VMImage cmdlet.
	-  Include the publisher, offer, SKU, and version for the VM image. These parameters are used by Azure Resource Manager templates that reference the VM image.
	-  Specify osType as Windows or Linux.
	-  Include your Azure Active Directory tenant ID in the form *<mydirectory>*.onmicrosoft.com.
	-  The following is an example invocation of the script:

```powershell
Add-VMImage -publisher "Canonical" -offer "UbuntuServer" -sku "14.04.3-LTS" -version "1.0.0" -osType Linux -osDiskLocalPath 'C:\Users\<me>\Desktop\UbuntuServer.vhd' -tenantID <mydirectory>.onmicrosoft.com
```

Note: The cmdlet requests credentials for adding the VM image. Provide the administrator Azure Active Directory credentials, such as *<Admin Account>*@*<mydirectory>*.onmicrosoft.com, to the prompt.

The command does the following:
- Authenticates to the Azure Stack environment
- Uploads the local VHD to a newly created temporary storage account
- Adds the VM image to the VM image repository
- Creates a Marketplace item

To verify that the command ran successfully, go to Marketplace in the portal, and then verify that the VM image is available in the **Virtual Machines** category.

##Remove a VM Image with PowerShell
Run the below command to remove an uploaded VM image. After removal, tenants will no longer be able to deploy virtual machines with this image.

```powershell
Remove-VMImage -publisher "Canonical" -offer "UbuntuServer" -sku "14.04.3-LTS" -version "1.0.0" -osType Linux -tenantID <mydirectory>.onmicrosoft.com
```

Note: This cmdlet does not remove any Marketplace item created as part of uploading a VM Image. These Marketplace items will need to be removed separately.

redpajama_set_name: RedPajamaGithub


Golar LNG's recent success in firming up plans for a Floating LNG project offshore Cameroon is already opening up doors to other potential stranded gas developments in the region, it has reported.
Under its 'GoFLNG' banner the company says its "tangible progress" with the Cameroon project (see DI, 19 January 2015, page 7) has "stimulated other interest in the region for fast-track, lower cost base, lower risk LNG solutions. Near-term effort is focused on West African opportunities for the deployment of the Golar Gimi from 2018".
The company says that with the attributes of these projects, being generally clean, relatively dry gas, located near-shore or in areas of relatively benign met-ocean conditions, the board has "identified several interesting employment opportunities for Gimi, but much work remains to be done before the vessel can be fully committed".
Some of the projects have potential capacity for more than one GoFLNG unit, it adds.
In January Golar LNG signed a Heads of Agreement (HOA) with Cameroon's Societe Nationale de Hydrocarbures (SNH) and Perenco Cameroon for the development of a FLNG export project located 20 km offshore utilizing Golar's GoFLNG liquefaction technology. The HOA is based on the allocation of 500 Bcf of gas reserves from the Kribi fields, to be exported to global markets via the first GoFLNG facility Golar Hilli, under construction at Keppel's shipyard in Singapore. Production is anticipated to be around 1.2 MMtonnes of LNG per annum over an approximate 8-year period.
Golar says that as of end-January the overall Hilli FLNG project progress was 30.3% complete against scheduled progress of 25.5% (4.8% ahead of schedule). Cumulative value of work done, excluding the value of the original vessel, amounted to US $305 million at year end, $52 million of that being incurred in Q4. The company says the Hilli project is "comfortably within budget and on track for commencement of operations by the targeted April 2017". It also added that the board and the partners have agreed a target date of June this year for the close-out of all technical and commercial matters pertinent to the fully termed agreements. Golar will be compensated on a fixed tariff basis for the FLNG unit.
Golar's above-mentioned Golar Gimi is to be a speculative conversion into a second FLNGV facility, again to be built by Keppel. The Gimi will be delivered to Keppel Shipyard in the second quarter of this year. Golar expects the conversion to be completed and delivered in approximately 33 months.
The company did add that, to retain flexibility in the roll-out of the GoFLNG strategy, it has secured beneficial cancellation provisions, which if exercised prior to November this year will allow termination of the Gimi contracts and the recovery of previous milestone payments, less a set cancellation fee.