Fast Analog Transmission for High-Mobility Wireless Data Acquisition in Edge Learning

By implementing machine learning at the network edge, edge learning trains models by leveraging rich data distributed at edge devices (e.g., smartphones and sensors) and in return endow on them capabilities of seeing, listening, and reasoning. In edge learning, the need of high-mobility wireless data acquisition arises in scenarios where edge devices (or even servers) are mounted on the ground or aerial vehicles. In this paper, we present a novel solution, called fast analog transmission (FAT), for high- mobility data acquisition in edge-learning systems, which has several key features. First, FAT incurs low-latency. Specifically, FAT requires no source-and-channel coding and no channel training via the proposed technique of Grassmann analog encoding (GAE) that encodes data samples into subspace matrices. Second, FAT supports spatial multiplexing by directly transmitting analog vector data over an antenna array. Third, FAT can be seamlessly integrated with edge learning (i.e., training of a classifier model in this work). In particular, by applying a Grassmannian-classification algorithm from computer vision, the received GAE encoded data can be directly applied to training the model without decoding and conversion. This design is found by simulation to outperform conventional schemes in learning accuracy due to its robustness against data distortion induced by fast fading.


I. INTRODUCTION
Envisioned as an evolution in computing, edge learning refers to the implementation of machine learning at the network edge so as to leverage enormous data distributed at edge devices (e.g., smartphones and sensors) for training models [1], [2]. Subsequently, the models are applied to empowering edge devices with the capabilities of seeing, listening and reasoning. While computing speeds are growing rapidly, the latency in wireless data acquisition has emerged to be the bottleneck of fast edge learning [2]. This issue is exacerbated in high-mobility scenarios where edge devices (or even edge servers) are mounted on ground or aerial vehicles as illustrated Fig. 1(a) [3]. High-mobility data acquisition faces several challenges: 1) robustness against fast fading, 2) low-latency given short connection time, 3 The design of the FAT scheme builds on three ideas from communication and learning. The first idea is analog transmission based on linear analog modulation that has been deployed previously in different settings such as fast transfer of channel-state information (CSI) [4] and over-the-air functional computation in sensor networks [5]. Compared with digital transmission, the analog design does not require source-end-channel coding and decoding, thereby reducing computation complexity. Moreover, direct transmission of analog data instead of a quantized bit stream shortens the transmission duration. In terms of learning performance, our findings suggest that a customized design of analog transmission targeting learning (e.g., FAT) can be more robust than digital counterparts against data distortion by fast fading at high mobility.
The second idea is blind multiple-input-multiple-output (MIMO) transmission without CSI.
This idea was first developed in the classic area of non-coherent MIMO, which is a digital spacetime modulation scheme [6], [7]. Its unique feature is a modulation constellation comprises a set of subspace matrices. The transmitted space-time symbol in the form of such a matrix is invariant to rotation by a block fading channel that remains constant within each symbol duration but varies over different durations. Thus the matrix can be transmitted and detected even without CSI at either side, referred to hereafter as the channel-invariant property [6]. Consequently, channel training is unnecessary, thereby reducing the transmission latency and overhead as illustrated in Fig. 1(b). On the other hand, non-coherent MIMO cannot support spatial multiplexing like its coherent counterpart. The resultant low data rates makes the former less popular in practice and its applications are limited to low-rate ultra-fast machine-type applications [3], [8]. In contrast, the proposed scheme retains the advantages of both technologies, namely channel-invariant property and (analog) spatial multiplexing. The said property is achieved by the proposed Grassmannian analog encoding (GAE), a key FAT component, which encodes a data sample (an analog vector)  The differences between FAT and conventional MIMO schemes are summarized in Table I.
In this paper, we consider a typical edge-learning task of training a classifier model. The last idea pertains to edge learning and is to apply a Grassmann classification algorithm for classifying the received GAE encoded training data. Such algorithms were originally developed for computer vision where image features or motions are represented as subspaces or equivalently points on a Grassmann manifold, referred to as Grassmann data [9]. Via the application of such an algorithm, classification can be seamlessly integrated with FAT since the received GAE encoded data can be directly used in learning without decoding and conversion. Furthermore, the integration leads to accurate edge learning with robustness in data acquisition against fast fading.
In summary, an edge learning system based on FAT comprises the following three components (elaborated in Section III).
1) Grassmann analog encoding: At each edge device, the proposed GAE encodes data samples into subspace matrices by projection onto a Grassmann manifold to enable blind MIMO transmission and robust edge learning.
2) Analog transmission and detection: The GAE encoded data is transmitted using linear analog modulation and blindly detected at the edge server without channel knowledge.
3) Edge learning: At the edge server, the received Grassmann data is used for training a classifier model using a Grassmann-classification algorithm from computer vision.
By evaluating the classification performance of a model and transmission latency using simulation (see Section V), the proposed FAT scheme is found to substantially outperform the conventional coherent (analog and digital) MIMO transmission at high mobility.
Trained models Figure 2. An edge-learning system based on FAT.

A. System Model
Consider the edge-learning system illustrated in Fig. 2 where an edge server trains a classifier using a training dataset transmitted by multiple edge devices. The transmissions by devices are based on time sharing and independent of channels given no CSI. All nodes are equipped with antenna arrays, resulting in a set of narrow-band MIMO channels. Let N t and N r denote the numbers of transmit and receive antennas, respectively. Time is divided into baseband sampling intervals, called (time) slots. Then the slot-t realization of the MIMO channel from an active device to the server can be represented by the N r ×N t matrix H t . Given an analog vector-symbol g t transmitted by the active device, the received signal at the server is where P is the transmission power and w t the additive-white-Gaussian-noise (AWGN) vector.
In this work, we focus on transmission of data samples that dominates the data acquisition process. Their labels have finite values and naturally can be transmitted using digital non-coherent MIMO modulation over a low-rate channel, called label channel, orthogonal to the high-rate data channel. Due to its low rate, the label channel can be reasonably assumed to be noiseless similarly as the CSI feedback channel [10].

B. Simulation Models
Simulation for evaluating learning performance is based on the following data and channel models. The data at different edge devices are assumed to be independent and identically distributed (i.i.d) based on the classic mixture of Gaussian (MoG) model, which is widely adopted in the machine-learning literature. Each data sample is a 1-by-L complex random vector. Let M denote the number of data classes. Then the i-th sample, denoted as s (i) , from the m-th class can be modelled as where µ m is the mean of the m-th class and z (i) ∈ C 1×L a deviation vector comprising i.i.d.
CN (0, σ 2 s ) elements. Next, high mobility induces temporally correlated MIMO channels. Assuming rich scattering, the classic Clark's model is applied that translates a speed into the level of channel temporal correlation. Specifically, within the duration of transmitting a data sample, two realizations of the (m, n)-th coefficient of the channel H t separated by τ slots are correlated with the correlation function given as where f D = fcv c with v being the speed, f c carrier frequency and c speed of light, and J 0 is the zero-th order Bessel function of the first kind.

III. FAST ANALOG TRANSMISSION SCHEME
In this section, we discuss two key algorithms in the proposed FAT scheme, namely GAE and blind analog transmission and detection (see Fig. 2). The received Grassmannian dataset is used for training a classifier model using an existing Grassmannian classifiaction algorithm such as sample Karcher mean [11], which is adopted in simulation. The details are omitted for brevity.

A. Grassmann Analog Encoding
To facilitate exposition, some mathematical notions are defined as follows. The (n, m) Grassmann manifold is a set of all m-dimensional subspaces in C n , denoted by G n,m [12]. For the special case of G 3,1 , each point on the manifold geometrically corresponds to a unique line passing through the origin as illustrated in Fig. 3. For ease of notation, a point on G n,m that is a subspace is usually represented by an arbitrary basis matrix spanning the subspace, denoted as Υ. The subspace distance between two points Υ and Υ on the Grassmannian G n,m , denoted as d p (Υ, Υ ), is measured using the commonly used metric of Procrustes distance for its better performance in simulation: As discussed, GAE at the active device endows on FAT the channel-invariant property, thereby enabling blind analog transmission with robustness against fast fading. As illustrated in Fig. 3, the mathematical principle of GAE is to project original data samples (vectors in the Euclidean space) onto the Grassmann manifold, generating subspace matrices as the output. The GAE algorithm is described as follows.

Data in the Euclidean space
Data on the Grassmannian Grassmann manifold Figure 3. Principle of Grassmann analog encoding.
Step 1 (Vector-to-matrix conversion): Consider a data sample that is a 1 × L row vector, say s (i) , with L being an integer multiple of the number of transmit antennas N t . Then s (i) can be divided into 1 × T sub-vectors with T = L/N t : . It follows that s (i) can be converted into a N t × T data matrix X (i) having {s (i) n } as rows. The matrix X (i) such constructed is typically fat (N t < T ) since a data-sample vector is usually long (L N t ). For the case where L is not an integer multiple of N t , zero-padding can be applied to lengthen s (i) so that the integer-multiple constraint is met.
Step 2 (Projection onto Grassmannian): The key step in encoding is to project the matrix X (i) constructed in the preceding step onto a single point on the Grassmannian G T,Nt . To this end, decompose the matrix X (i) by singular value decomposition (SVD) as Then G (i) is a N t × T basis matrix spanning the row space of X (i) and thus a point on G T,Nt .
The encoder uses G (i) as the output from encoding the data-sample s (i) .
In summary, the advantages of GAE are threefold: 1) enabling blind transmission and detection as discussed in the next sub-section, 2) endowing on edge learning robustness against data distortion by fast fading as shown in simulation results, and 3) allowing seamless integration with learning on Grassmannian without signal decoding and conversion.

B. Blind Analog Transmission and Detection
Given Grassmann encoding, the procedures for subsequent blind analog transmission and detection in FAT are described as follows.
Step 1 (Analog transmission): After encoding each data-sample, say s (i) , into a subspace basis matrix G (i) , the N t × T matrix G (i) is directly transmitted by the active device over T slots using linear analog modulation and the array of N t antennas. To reflect channel temporal variation, it is necessary write G (i) in terms of its columns: T ]. Then the received signal due to the transmission of G (i) can be represented by the N r × T matrix For continuous time-shared distributed uploading of total N data samples, t = 1, 2, · · · N T . It is important to observe from (5) that due to high mobility, the channel {H (i) t } varies in the T -slot transmission duration of a single data sample, which has a negative effect on decoding as discussed in the sequel.
Step 2 (Grassmann analog detection): The detection of the transmitted encoded analog space-time symbol G (i) involves the extraction of the row space, denoted by the N t × T unitary matrix G (i) , from the SVD of the received N r × T space-time signal Y (i) specified in (5), Consider the special case of zero noise and static channel. The In other words, G (i) and G (i) are the identical point on the Grassmannian, corresponding to perfect detection.
In the presence of noise and channel variation, they are two different points and the resultant detection error affects learning. Based on the above detection procedure, the output training dataset, called Grassmann dataset, is a sequence of N labeled subspace matrices (points on the Grassmannian), [ G (1) , G (2) , · · · , G (N ) ], whose labels are acquired by the server via the said low-rate label channel.
Remark 1 (Blind Transmission and Detection). Both the analog transmission and detection in the above steps are independent of the channel. In particular, the detection of Grassmann dataset involves SVDs of the received array observations that do not require any channel knowledge.

A. Grassmann Analog Encoding Preserves Clustering
An important reason FAT supports edge classification is that GAE retains the class structure in the original dataset. This property is illustrated in Fig. 4 where the high-dimensional datasets are visualized in the 2D plane using a well known visualization algorithm, t-distributed stochastic neighbour embedding (t-SNE). As discussed in the sequel, GAE incurs DoF loss in the dataset.
Consequently, one can observe form Fig. 4 that data classes are less compact after GAE, sacrificing some level of discriminant of the dataset. The loss, nevertheless, yields communication advantages discussed shortly.

B. Trading DoF Loss for Robustness and Low Latency
The Grassmann encoding design leads to the DoF loss (or discriminant loss), which may make data points among different classes that are well-separated in the Euclidean space become much closer or even overlapped on the Grassmannian. The loss has a negative effect on the classification performance. The phenomenon is illustrated by the following example. Besides SVD, an alternative method for GAE is LQ decomposition. Consider the LQ decomposition of two data matrices X = L X G X and Y = L Y G Y , where the unitary matrices G X and G Y represent identical subspaces (or identical encoding outputs) as the SVD counterparts and L X and L Y are lower triangular matrices. If span(G X ) = span(G Y ) but L X = L Y , the Euclidean distance between X and Y is d 2 E (X, Y) = 0. However, based on (4), the Procrustes distance between the GAE encoded data samples, namely G X and G Y , is d 2 p (G X , G Y ) = 0. A key finding in this work is that the DoF loss of GAE is more than compensated by its robustness against fast fading that can cause severe errors in data transmission without GAE.
As a result, GAE leads to a net performance gain over conventional schemes at high mobility. The performance of FAT is benchmarked against two high-rate coherent schemes: digital and analog MIMO transmission, both of which assume a MMSE linear receiver and thus require channel training to acquire the needed CSI. Like FAT, analog MIMO transmits data samples directly by linear analog modulation. On the other hand, digital MIMO quantizes data samples into 8-bit per coefficient and modulates each symbol using QPSK before MIMO transmission.
All considered schemes have no error control coding.

A. Communication Latency Performance
While FAT is free of channel-training, benchmark schemes incur training overhead that can be quantified by the fraction of a frame allocated for the purpose i.e., the ratio P/(P + D) with P and D illustrated in Fig. 1(b). The curves of overhead versus Doppler shift are displayed in Fig. 5 for FAT and two mentioned benchmarking schemes for a given classification-error rate of 1 × 10 −3 . One can observe that the overhead grows monotonically with the Doppler shift as the channel fading becomes faster. For high-mobility with Doppler approaching 10 −2 , the overhead can be more than 12% and 6% for digital and analog coherent MIMO, respectively. Furthermore, given the same performance, digital coherent MIMO (with QPSK modulation and 8-bit quantization) requires 4 times more frames for transmitting the training dataset than the two analog schemes. This suggests that analog transmission is preferable for data acquisition targeting edge learning.

B. Learning Performance
Classifier models discussed in Section III are trained using the training dataset acquired using different transmission schemes and then evaluated using the test dataset. The resultant classification error rates are compared in Fig. 6 by varying Doppler shift and average transmit SNR. Several observations can be made. In the range of moderate to large Doppler shift (i.e., larger than 6 × 10 −3 ), the proposed FAT outperforms the benchmarking schemes, supporting the former's intended application in high-mobility data acquisition. Furthermore, at high mobility (i.e., Doppler equal to 0.01), FAT achieves the best performance in the practical SNR range (0 − 15 dB). On the other hand, analog and digital coherent MIMO are preferred at low and high SNRs, respectively. The above observations reconfirm the conclusion from preceding latency comparison that analog transmission (especially FAT) is a promising solution for high-mobility data acquisition for edge learning.