Intelligent Spectrum Learning for Wireless Networks with Reconfigurable Intelligent Surfaces

Reconfigurable intelligent surface (RIS) has become a promising technology for enhancing the reliability of wireless communications, which is capable of reflecting the desired signals through appropriate phase shifts. However, the intended signals that impinge upon an RIS are often mixed with interfering signals, which are usually dynamic and unknown. In particular, the received signal-to-interference-plus-noise ratio (SINR) may be degraded by the signals reflected from the RISs that originate from non-intended users. To tackle this issue, we introduce the concept of intelligent spectrum learning (ISL), which uses an appropriately trained convolutional neural network (CNN) at the RIS controller to help the RISs infer the interfering signals directly from the incident signals. By capitalizing on the ISL, a distributed control algorithm is proposed to maximize the received SINR by dynamically configuring the active/inactive binary status of the RIS elements. Simulation results validate the performance improvement offered by deep learning and demonstrate the superiority of the proposed ISL-aided approach.


I. INTRODUCTION
Due to the dynamic nature of the wireless environment that results in severe signal fluctuations caused by multipath fading and the presence of large obstacles, the wireless link between a desired cellular user and a base station (BS) may not be reliable enough or may even undergo a complete outage. To tackle this issue, reconfigurable intelligent surfaces (RISs) have been proposed to improve the received signal-to-interference-plus-noise ratio (SINR) at the users by appropriately reflecting the incident signals and generating directional beams [1].
In the literature, some preliminary works investigated the optimization of RIS-assisted wireless communications. In [2], a joint transmit power allocation and phase shift design was developed to maximize the energy efficiency. In [3], the authors considered a downlink RIS-assisted multiuser communication system and studied a joint transmission and reflection beamforming problem to minimize the total transmit power. In [4], the channel estimation problem was investigated for an RIS-aided wireless communication system by jointly optimizing the training sequence of the transmitter and the reflection pattern of the RIS.
Furthermore, an RIS-assisted anti-jamming solution was proposed for securing wireless communications via reinforcement learning [5]. In [6], the authors introduced RISs in mobile edge computing systems, where a joint design of computing and communications was developed to minimize the computational latency.
However, most of the existing works assume either that no interference exists, which rarely occurs in practice, or that the interference is known and can be taken into account, which is not trivial to estimate since the interference is usually dynamically changing. These issues are exacerbated in RISaided systems, since they are nearly-passive surfaces with no active sensing capabilities for channel and interference estimation [7]. In order to elucidate the problem at hand, let us consider the case study depicted in Fig. 1(a), in which we consider a practical scenario where all the RISs serve one desired user (e.g., U 1 ). In the analyzed case study, the other users (e.g., U l , l ∈ [2, L]) are considered as interfering users for U 1 at each RIS. As a result, the signal reflected by each RIS is a mixture of the desired signal from U 1 and the interfering signals from the other users. In this context, the received SINR at the BS via an RIS that does not account for the interfering signals could be even worse than the SINR of the direct link. This impact of the interference is, in particular, more severe if the interfering devices are close to the desired user, e.g., in Fig. 1(a) U 2 may cause severe interference to U 1 at RIS 1 when the angle between them (i.e., λ) is small.
To overcome these challenges, in this paper, we empower a conventional RIS-assisted wireless system with 'intelligent spectrum learning (ISL) capabilities', by leveraging appropriately trained convolutional neural networks (CNN) at the RIS controller in order to predict/estimate the interfering devices from the incident signals, as highlighted in Fig. 1(b). In the proposed system, the active-inactive (or ON-OFF) status of each RIS 1 and the corresponding phase shifts need to be carefully optimized, since the interference distribution at each RIS is, in general, different. The corresponding SINR maximization problem turns out to be a mixed-integer nonlinear program (MINLP), which is usually difficult to solve. To tackle this issue, we decompose the original problem into two subproblems, which are solved independently and in a distributed manner. The proposed solution equips a conventional RIS with the capability to dynamically 'think-and-decide' whether reflecting or not the incident signals through the proposed distributed ISL principle.

II. SYSTEM MODEL AND PROBLEM FORMULATION
We consider an RIS-assisted uplink wireless system that consists of one BS, a set K of K RISs, and a set L of L users, where K ≥ 1 and L > K usually hold. We assume that the BS allocates all the RISs to serve one user at a time in order to improve the quality of the wireless link. Also, the direct links between the users and the BS are available. Under these assumptions, all the other users act as interferers for the intended user either through the direct links or through the links reflected by the RISs.
Each RIS operates as a nearly-passive surface, i.e., the RIS elements are passive but the RIS controller may consume power [8]. In particular, the RIS controller is equipped with a CNN, as shown in Fig. 1(b).
The CNN is discussed in the next sections.

A. RIS-Assisted Communication Model
Each RIS, denoted as R k , k ∈ K, is equipped with N k reflecting elements, which can be appropriately configured by the RIS controller to reflect the signals of the users towards the BS. In general, the RISs can be appropriately deployed so that line-of-sight (LoS) links can be established with the BS and, possibly, the users. We assume that the channel state information (CSI) of all channels involved is perfectly known at the BS 2 , which, in turn, can feed back the CSI to the RIS controller via a dedicated control channel [3], [6].
Each RIS can be in two possible states: ON and OFF. We introduce a binary variable β k ∈ {0, 1} to indicate the ON-OFF state of R k . β k = 1 indicates that R k is ON, which means that it reflects the incident signals, while β k = 0 indicates R k is OFF, which means that it does not reflect any signals.
As for the kth RIS that is ON, the amplitude reflection coefficient is assumed to be equal to one for all the N k reflecting elements and the phase reflection matrix is where Θ k = θ k 1 , θ k 2 , ..., θ k Nk denotes the vector of phase shifts that can be optimized by R k .

B. Wireless Channel Model
We consider an RIS-aided uplink wireless system, where the channels from the desired user (denoted as U l ) to R k , from R k to the BS, and from the mth interfering user (denoted as U m , m ∈ L, m = l) to R k , are h l,k ∈ C Nk×1 , g k ∈ C 1×Nk , and h m,k ∈ C Nk×1 , respectively. The channel gains of the direct links from U l to BS and from U m to BS are denoted by h d,l and h d,m , respectively. These channels are assumed to be perfectly estimated and quasi-static, hence remaining nearly-constant during the transmission time [3].
Without loss of generality, we assume that the users are randomly distributed and have time-varying traffic demands. This implies that the users may not be all active during the considered transmission time 3 . In particular, the total number of interfering users for U l is given by ω l = m∈L,m =l α m , where α m ∈ {0, 1} is a binary variable that indicates that U m is active (α m = 1) or inactive (α m = 0) and therefore can cause or not interference to U l , respectively. Accordingly, the set of active interfering users is W = {U m |α m = 1}, where ∀m = l and ∀m ∈ L.
Since the number of active interferers is dynamic, it is not easy to estimate the interference distribution over time. The received signal at the BS depends on the desired signal sent from U l (including the direct link and the link reflected by the RIS), the interference from the interfering users in W, and the white Gaussian noise: Interference from other users where p l and p m denote the transmit power of U l and U m , respectively, s l and s m are the unit-power information signals sent from U l and U m , respectively, and n l ∼ CN (0, σ 2 ) is the white Gaussian noise.
In addition, ξ k m,l ∈ [0, 1] accounts for the impact of the interference caused by U m to U l and that is associated to the kth RIS, as clarified in Assumption 1. Assumption 1. We assume that the impact of the strength of the interference reflected by an RIS is inversely proportional to the difference of the angles of incidence between the desired user and the interfering users at the RIS (see λ in Fig. 1(b) for an example).

C. Problem Formulation and Analysis
Let B = {β 1 , β 2 , ..., β K } denote the RIS binary decision vector that collects the binary variables that identify the ON-OFF status of the RISs. The set of RISs that are active is R = {R k |β k = 1} , ∀k ∈ K.
Accordingly, the SINR at the BS for the intended user U l is Our objective is to maximize the SINR in (3), by optimizing the RIS binary activation vector (B), and the phase shifts matrix of the active RISs which is denoted as To this end, we need to solve the following optimization problem:  Constraint (4b) indicates that the kth RIS can only be ON (i.e., β k = 1) or OFF (i.e., β k = 0) at one time. Constraint (4c) indicates that each RIS reflecting element can only provide a phase shift θ i n ∈ [0, 2π) without amplifying the signals.
Our Proposal: We observe that P 1 is an MINLP, which is NP-hard and whose global optimal solution is, in general, difficult to obtain. In addition, traditional optimization methods may be computationally intensive. An emerging approach to tackle this issue is to apply deep learning methods to solve P 1 at a reduced computational complexity [10], [11]. P 1 , however, may not be easy to solve even using deep learning methods, since the number of interfering users is a random variable that is unknown. In addition, during the channel estimation phase, the RISs cannot estimate on their own the active interferers because they only reflect the incident signals in a passive manner. This makes the solution of P 1 even more difficult. To address this challenge, we propose a distributed control mechanism that solves P 1 with the aid of the ISL algorithm.

III. DISTRIBUTED RIS CONTROL VIA INTELLIGENT SPECTRUM LEARNING
In this section, the ISL-aided distributed RIS control mechanism is introduced. The proposed approach leverages appropriately trained CNNs at the controller of the RISs, which can identify the active interfering users in a distributed way.

A. Intelligent Spectrum Learning
ISL is a multi-class classification algorithm that, based on a CNN, returns the set of interfering users for each intended user U l . To design the ISL algorithm, three main aspects need to be discussed: 1) RF traces collection, 2) offline CNN training, and 3) online CNN inference.
1) RF Traces Collection: As far as the RF data collection phase is concerned, historical RF traces are collected using a universal software radio peripheral (USRP2) testbed, which is wired connected (e.g., Gigabit Ethernet) to a host PC with an implementation of the GNU Radio, as illustrated in Fig. 2(a). In particular, the users are emulated through a laptop that is mainly responsible for baseband processing while a USRP2 platform is used for the up-conversion, the digital-to-analog (D/A) conversion, and wireless transmission of the signals. As far as the BS is concerned, another USRP2 module first receives the signals from the radio interface and then performs A/D and down-conversion. Subsequently, the laptop receives the signals from the USRP2 via the Ethernet and executes the baseband processing. Finally, the Inphase (I) and Quadrature (Q) sequences are stored as a file. In particular, the experimental setup for RF data collection using the USPR2 is performed by using signals at 2.4 GHz carrier frequency with 1 MHz bandwidth. In order to collect realistic RF signals in the presence of interference, we let multiple USRP2 units transmit RF signals to an USRP2 receiver. The RF traces have been collected as I/Q sequences, by including the wireless channel, for a wide range of SNR (e.g., from 0 to 20 dB with interval of 5 dB) in order to account for different interfering cases [12].
2) CNN Offline Training: The acquired RF traces have been used for training the CNN architecture illustrated in Fig. 2(b), where each convolutional layer is followed by a rectified linear units activation function for feature extraction. Fully-connected (FC) layers are used to classify the signals by using the softmax activation function for the output layer [13]. The training algorithms is based on the Adam algorithm that uses the cross entropy as the loss function. The CNN model is trained offline using TensorFlow on a GPU cluster (NVIDIA Tesla P100-PCIE-16GB).
Even though the training of the CNN does not account for all possible channel conditions, the generalization property of deep learning enables the trained CNN to infer channel conditions not included in the training dataset [14]. It is also noteworthy that several methods have been proposed to scale up the training process of deep neural networks across GPU clusters, which helps to further reduce the runtime of the offline training. Once the CNN model is appropriately trained, it can directly infer incident signals in near real-time. In other words, the proposed ISL-based framework moves the complexity from online computation to offline training.

3) CNN Online Inference:
The CNN is, in particular, trained in order to return the set of active interfering users based on different input signals at each RIS. Specifically, the received RF signals first undergo A/D conversion and frequency down-conversion. Then the baseband I/Q sequences are fed into the trained CNN model to perform online inference at the RIS controller.
By performing feed-forward calculation via the CNN model (i.e., online inference), the intefering users set for the desired user U l is obtained as {U m | α m = 1}, ∀m ∈ L, ∀m = l, If ω l ≥ 1,

∅,
If ω l = 0, where α m denotes the inferred state flag of U m , and ω l indicates the inferred total number of intefering users in the incident signal.

4) An Illustrative ISL Example:
To better understand the proposed ISL algorithm, we illustrate an example with only two users (denoted as U 1 and U 2 ), and each user has a binary state, i.e., 'active'/'ON' and 'inactive'/'OFF'. In this case, there exist four combinations of signals from the perspective of each RIS: (1) 'Idle' (indicating that both U 1 and U 2 are inactive), (2) 'Only U 1 ' (indicating that only U 1 is active), (3) 'Only U 2 ' (indicating that only U 2 is active), and (4) 'U 1 +U 2 ' (indicating that both U 1 and U 2 are active). Based on the superimposed incident signal(s), the RIS needs to identify the composition of the signal(s), i.e., to identify the correct class out of the four possible classes of signals. Therefore, this signal identification boils down to a four-class classification problem, as illustrated in Table I.

1) Optimal Phase Shifts Calculation:
To calculate the phase shifts at the RIS, we denote the obtained CSI associated to the kth RIS as C k = {h l,k , h m,k , g k , h d,l , h d,m }. Based on the inferred interfering users (including I l and ω l ) obtained via the trained CNN, the received SINR of the signal sent from U l is given by Based on (6), we first obtain the phase shifts of each RIS under the assumption β k = 1, ∀k ∈ K, and then optimize the optimum binary ON-OFF vector B based on the obtained phase shifts. The first step, in particular, can be formulated as We observe that P 2 is a non-convex problem, which can be tackled by using several methods, such as the semidefinite relaxation (SDR) method [3] and the successive convex approximation (SCA) method [15].
The optimal solution for the kth RIS is denoted by Θ * k , and the corresponding reflection-coefficient matrix is Φ * k . With the obtained reflection-coefficient matrix, each RIS ON-OFF status is optimized via the following RIS binary control algorithm.
2) Distributed RIS Binary Control Algorithm: Assuming that only the kth RIS is ON, i.e., β k = 1, the received SINR of the signal sent from U l to the BS via the kth RIS is If, on the other hand, all the K RISs are OFF, i.e., β k = 0 for k ∈ K, the received SINR of the signal sent from U l to the BS via the direct link is Based on (8) and (9), the kth RIS decides whether to be ON or OFF as detailed in Remark 1.

Remark 1.
The kth RIS should be ON if γ k l ≥ γ D l holds. This indicates, in fact, that the desired signal from U l can be enhanced via the kth RIS. Otherwise, the kth RIS should be OFF to avoid the degradation of the desired signal. In this case, the incident angle between the signals of the desired user and the interfering signals at the RIS is in general small.
The approach for solving P 2 is summarized in Algorithm 1, which is executed at each RIS controller when the incident signal is received. Specifically, upon receiving the incident signal, each RIS controller identifies the set of interfering devices by extracting the I/Q samples from a copy of the incident signal and feeding them into the trained CNN. Based on the classification outcome of the CNN, the RIS controller can set the RIS ON-OFF state in a distributed manner.
3) Computational Complexity: The total computational complexity includes the online inference via the CNN and the iterative algorithm to solve the phase shift optimization problem P 2 and the RIS ON-OFF optimization problem.
• Complexity for CNN online inference: The CNN is trained offline in a supervised fashion, therefore the complexity of training can be ignored. The trained CNN model has a quadratic time complexity Calculate Θ * k and Φ * k by solving problem P 2 ; 5: Calculate γ k l and γ D l via (8) and (9), respectively; 6: if γ k l ≥ γ D l then 7: Keep the kth RIS ON, i.e., β k = 1; Add the kth RIS binary decision β k to B. • Complexity for solving the phase shift optimization problem P 2 : To solve the problem P 2 , the complexity lies in computing the optimal phase shift at each iteration of the optimization method, e.g., the SCA method [15] whose complexity is O(Qz), where Q = K k=1 N k denotes the total number of elements of all the RISs, and z is the total number of the iterations required.
• Complexity for solving the RIS ON-OFF optimization problem: Since γ k l and γ D l need to be calculated via (8) and (9), respectively, the computational complexity of solving the RIS ON-OFF optimization problem required at each RIS controller is O(K).
As a result, the total complexity is O(M 2 CK + Qz + K), which grows linearly with the total number of RISs.

IV. SIMULATION RESULTS
In this section, we first evaluate the inference accuracy of the trained CNN model and the computational complexity of the proposed ISL-based DRBC algorithm. Then we validate the benefits of deploying the ISL-based DRBC algorithm.

B. Computation Time
The proposed ISL-based DRBC algorithm allows us to obtain the optimal ON-OFF status of each RIS.
As detailed in previous text, the optimal ON-OFF status of the RISs can be formulated as the solution of a non-convex MINLP, which is usually challenging to solve [15]. In this section, we compare the proposed ISL-based DRBC algorithm against the spatial branch and bound (sBB) method, which is often employed to solve non-convex MINLP [16], in terms of computation time.  Table III. Compared to the traditional sBB method, the ISL-based DRBC algorithm results in much lower computation time while still yielding the optimal ON-OFF status for each RIS.
The computation time of the proposed algorithm is less than one-thousandth of the computation time of the sBB method when the number of RISs varies from 2 to 5. The incident angle between the BS and U 1 at the RIS is 150 o , and the incident angle between U 2 and We assume that ξ k m,l is linearly inversely proportional to λ. The channel parameters are selected according to the 3GPP Urban Micro standard [17], which describes the path loss for both line-of-sight and non-line-of-sight components [18]. The transmission power of U 1 is 20 dBm, the noise power σ 2 is -94 dBm, the carrier frequency is 3 GHz, and the reflection amplitude is equal to one.
2) Simulation Results: We evaluate the performance of the proposed ISL-aided algorithm by comparing it with two benchmarks: 'RIS always ON' and 'RIS always OFF'. Figs. 3(a)-(b) depict the achievable SINR versus the angle of incidence (λ) for K = 1 and p m = 10, 15, 20 dBm. We observe that the SINR first gradually decreases as λ increases due to the reduction of the distance between U 2 and BS, and then increases due to the perfect interference elimination at the RIS. When λ is small in particular, the RIS is prone to be OFF since the interference reflected via the RIS is more pronounced. If λ is large, on the other hand, the impact of the interference is reduced and it is more probable that the RIS is ON. In general terms, however, the impact of λ on the system performance is still an open issue, whose analysis is postponed to a future research work.
In Figs. 4(a)-(c), the SINR versus K is illustrated, where p m = 10 dBm, the distance between U 2 and the RIS is 5 m. We observe that the RISs are always OFF if λ = 0, since the impact of the interference is too high. If λ is very large, e.g., λ = 150 o in Fig. 4(b), the impact of the interference is low and the RISs are always ON. When λ is randomly selected, e.g., λ ∈ [30 o , 120 o ] in Fig. 4(c), the SINR  obtained by the ISL algorithm increases with K and outperforms the two benchmarks, by about 100% with respect to 'RIS always OFF' and by nearly 300% with respect to 'RIS always ON' when K = 5.
From Figs 4(a)-(c), we conclude that the performance of the proposed ISL algorithm largely depends on λ, which impacts the interference cancellation at the RISs.

V. CONCLUSION AND FUTURE WORK
In this paper, we introduced an ISL algorithm that uses appropriately trained CNNs for the interference management in RIS-aided multi-user uplink networks. With the aid of the ISL algorithm, the RISs are capable of inferring the interfering signals directly from the incident signals. A distributed control algorithm was proposed to maximize the received SINR by dynamically configuring the binary status of the RIS elements. Simulation results validated the performance improvement offered by the proposed ISL-aided RIS approach. Offline training is only a candidate way to train a CNN, which may need to be retrained when RF data distribution changes significantly. This issue may be avoided by using online training methods, such as federated learning, which is a promising method for application in dynamical wireless environments.