extfieldsofstudy: Mathematics Physics

text:
Control energy scaling in temporal networks

In practical terms, controlling a network requires manipulating a large number of nodes with a comparatively small number of external inputs, a process that is facilitated by paths that broadcast the influence of the (directly-controlled) driver nodes to the rest of the network. Recent work has shown that surprisingly, temporal networks can enjoy tremendous control advantages over their static counterparts despite the fact that in temporal networks such paths are seldom instantaneously available. To understand the underlying reasons, here we systematically analyze the scaling behavior of a key control cost for temporal networks--the control energy. We show that the energy costs of controlling temporal networks are determined solely by the spectral properties of an"effective"Gramian matrix, analogous to the static network case. Surprisingly, we find that this scaling is largely dictated by the first and the last network snapshot in the temporal sequence, independent of the number of intervening snapshots, the initial and final states, and the number of driver nodes. Our results uncover the intrinsic laws governing why and when temporal networks save considerable control energy over their static counterparts.

Yet most existing studies of controllability have been premised on static networks [7][8][9][10][11][12][13][14][15][16][17], with comparatively limited attention devoted to the case of (discrete-time) dynamics on temporal networks [18,19]. Putatively static networks are often aggregated from an underlying temporal sequence of snapshots, representing subsets of nodal interactions active at any given time. With this recognition that temporal networks are in many areas the rule rather than the exception, many studies have explored temporal analogues of important structural features of static networks including the smallworld [20] and scale-free [21] properties, and community structure [22]. But the temporal nature of networks cannot be neglected for many dynamical processes on networks either [23][24][25][26][27][28][29]. Indeed, consider that if Alice interacts with Charlie after first interacting with Bob, then information (or a virus) cannot be propagated from Charlie to Bob through Alice. The effects of such timing constraints on system dynamics have been reported on accessibility [30], diffusion or epidemic spreading [31][32][33][34], and human cooperative behavior on dynamical population structures [35].
Recent research has revealed that control, too, is a dynamical process profoundly affected by network temporality, and in a surprising way [36]. It has been shown that temporal networks enjoy control costs orders of magnitude lower compared to their static counterparts [36]. Yet, the laws governing the control costs for temporal networks remain elusive. Here we focus on the behavior of one key control cost-the control energy-to control temporal networks, deriving a simple rule that governs the scaling of the control energy with the dynamical evolution of the network topology.

II. CONTROL ENERGY
We regard a temporal network as an ordered sequence of M separate networks called snapshots on a fixed set of N nodes, and we denote by A m the adjacency matrix of snapshot m for m = 1, 2, · · · , M . Starting from the first snapshot at time t 0 , we assume each snapshot m lasts for a duration of τ m time units. We consider networks whose dynamical state followṡ over the time interval t ∈ [t m−1 , t m−1 + τ m ), where t m−1 = j=m−1 j=1 τ j and x i (t) is the state of node i at time t with x(t) = (x 1 (t), x 2 (t), · · · , x N (t)) T ∈ R N . Here, u m (t) ∈ R p is a vector containing the p independent control inputs and B gives the (constant) mapping between these inputs and the driver nodes of the network-those that receive input directly. We will focus on the case where one input corresponds to one driver node, as has been the norm in previous studies of network control [11-13, 37, 38].
The canonical definition of the control energy required to drive a system from state x 0 at t 0 to x f at t f is 1 2 t f t 0 u T (t)u(t)dt, a definition that applies to arbitrary systems, whether linear or nonlinear, temporal or static [11,14,39,40]. In the case of a temporal network obeying Eq. (1), we have shown [36] that the corresponding energy-optimal control signal can be constructed piecewise as . This signal is parameterized by the constant vectors c m = (c m,1 , c m,2 , · · · , c m,N ) T ∈ R N , which are unique and can be calculated according to the quadratic programming problem:

III. BOUNDS OF THE OPTIMAL CONTROL ENERGY
One can solve (2) analytically and find that the minimal energy required to control a temporal network between initial state x 0 and final state x f is where W eff = SWS T . For a given pair of initial and final states, the control energy is thus determined by the spectral properties of the "effective" Gramian matrix W eff , analogous to the static network [11]. Henceforth, we will focus on the case x 0 = 0 (for the general initial states, please refer to the SI). By normalizing so that x f lies at unit distance we can consider the normalized control energy, Irrespective of the location of x f , this allows us to impose lower E and upper bounds E on the control energy as where η max and η min are the maximum and minimum eigenvalues of W eff . Since W eff is a real and symmetric matrix, all eigenvalues are real and the minimum and maximum are well-defined. Note that when all snapshots are identical, meaning the network structure is time-invariant, W eff reduces to the typical controllability Gramian for static networks [11] (for a proof of this, please see Sec. E in the SI). The above bounds apply to arbitrary temporal sequences, regardless of whether the dynamics of the constituent snapshots are stable, unstable, or a mix. This will allow us to systematically study the behavior of the control energy for a range of temporal networks and determine the regimes in which they have an advantage over their static counterparts.

IV. THE SCALING BEHAVIOR OF THE BOUNDS FOR TWO SNAPSHOTS
The lower (upper) bound E (E) of the optimal control energy indicates the best (worst) case control direction, that is, the direction of the eigenvector corresponding to the maximum (minimum) eigenvalue η max (η min ) of W eff . The properties of the corresponding eigenvalues in turn determine the scaling behavior of E and E. To understand the scaling behavior of E (E) with respect to the duration time h of each snapshot, we first analyze the case of two snapshots (A 1 , B) and (A 2 , B), and later generalize to an arbitrary number of snapshots. By approximating the maximum (minimum) min ) of A 1 and A 2 (see SI Sec. A), we can obtain an analytic prediction of the scaling behavior of the E (E) for controlling temporal networks from 0 to x f . Table I summarizes the possible behaviors of E, which we find is dominated by the maximum eigenvalue λ (2) max of the second snapshot A 2 for large h. In this regime, we can therefore separate the behavior of E into three cases based on the sign of λ where H(x) is the Heaviside step function, and λ (1) max is the maximum eigenvalue of the first snapshot and otherwise decreases exponentially with the exponent λ max . When h is small, the law of unique with E ∼ h −1 . These analytical predictions, which are summarized in Table I and Table II, are corroborated by numerical results (Figs. 1 and 2).
Here we have employed the Laplacian matrix with self-loops to represent the weighted undirected snapshot A m . This allows us to tune the values λ min τ m /τ ). And we assume all snapshots' durations are identical (τ m = h for all snapshots m) for simplicity. We have checked the robustness of our results for other settings of link weight.

V. THE FIRST AND LAST SNAPSHOTS DETERMINE THE SCALING BEHAVIOR
We can evaluate the contribution each snapshot makes to the overall control energy using the following expression (see SI Sec. G) We find that when we control a system from arbitrary x 0 to x f , it is the first and last snapshots that determine the scaling behavior of the control energy required (see SI Sec. G). This somewhat surprising result can be understood by which indicates that E * (x 0 , x f ) is dominated by A 1 and A M for any kind of inputs (this equation is derived by minimizing Eq. (G1)). Thus, although the whole sequence of snapshots influences the exact control signal u(t) and globally optimal trajectory x * (t), it is only the first and last snapshots that determine the corresponding control energy. This can be understood by the fact, that it is these snapshots from which the temporal network must "lift off" from x 0 and "land" at final state x f .

NUMBER OF SNAPSHOTS
Assuming for simplicity that the system starts at the origin (x 0 = 0), only the last snapshot matters because in principle, one can exploit the fact that until the final snapshot and from that point proceed to x f . In this case, inner snapshots merely contribute to the exponent that governs the exponential decrease of the energy for large h, where λ (m) max is the maximum eigenvalue of the snapshot A m . For E (for E, it is similar, and please see SI), when the last snapshot A M is not negative definite (λ (M ) max > 0), E will decrease exponentially with an exponent between λ (M ) max and λ max ; when A M is negative definite (λ (M ) max < 0), E will decrease from a constant to exponentially with exponent λ max ; when A M is negative semi-definite (λ (M ) max = 0), E will decrease hyperbolically first and eventually exponentially with rate λ max . Above analytical results are validated by numerical calculations (see Fig. 3, Fig. S2 and Fig. S4 in SI Sec. C). Finally, when h is small, we predict that E ∼ h −1 , which is confirmed numerically by simulations and shown in Fig. S1. The detailed analytical scaling behavior of E and E for arbitrary number of snapshots and driver nodes may be found in SI Sec. D.

VII. DISCUSSION
Our results provide a comprehensive anatomy of the control energy scaling for undirected temporal networks with respect to the stability properties of the underlying system matrices. Our results can readily be generalized to the case of weighted directed networks, provided the effective Gramian matrix is diagonalizable. In this case, the traditional eigenvalues would be replaced by the real parts of the new (now complex) eigenvalues. In the present work each snapshot is confined to be controllable, as it is difficult to perform a systematic equal comparison of the optimal control energy in temporal versus static networks if either of them is only partially controllable. This is true in part because the optimal control trajectory may be highly nonlocal even as the distance between x 0 and x f approaches zero [14,36].
The analysis of a single snapshot can provide intuition about why E and E are divided into three cases according to the properties of the final snapshot. For small h (high temporality), the system has less overall time to allocate its optimal control scheme, meaning the last snapshot has correspondingly less influence over the scaling of both E and E, thus explaining their broad power-law bahavior in this case. For a final state chosen randomly from the controllable space, it has been shown the minimum control energy to reach it is dominated by the upper bound at the same control distance x f − x 0 for both temporal and static networks [11,36]. Our discovery of the scaling behavior of both E clearly explains the previous discovery [36] that temporal networks require orders of magnitude less control energy than their static counterparts, especially in the regime of high temporality (small h).
Moreover, our analysis of E provide us the "best case" control scenario at a given control distance.
To gain a deeper understanding of the scaling behavior of control energy for temporal networks, we can consider E as an example. The optimal energy is inevitably affected by the internal system dynamics in the absence of control. Indeed, for a single snapshot, the autonomous dynamicsẋ(t) = A m x(t) will naturally facilitate movement away from the origin, when the system is unstable (A m is PD, i.e. λ (m) min > 0). It follows that, when external control inputs corresponding to the maximal energy are applied, the control trajectory corresponding to the optimal maximum energy E will choose the least hindrance from the internal dynamics, namely the control direction along the eigenvector of λ (m) min . It is the facilitation of the internal dynamics that leads to the exponential decrease of E over large control time h. When there exists at least one negative eigenvalue (say, λ (m) min < 0, meaning A m is NPD), the optimal control path will take advantage of this and drag E to a larger value even though the system is unstable along other eigenvectors. When λ (m) min = 0 (A m is PSD), E will correspond to a trajectory aligned with the eigenvector of λ (m) min even with other positive eigenvalues, leading to the hyperbolic decay of E. E can be similarly understood for long snapshot durations (low temporality) by virtue of the attributes of the spectral properties of the system matrix.
Temporal networks are known to possess tremendous flexibility over static networks precisely because they allow exploitation of the most favorable dynamical features of many networks (snapshots) as opposed to just one. Yet here, we have shown that the large-scale behavior of the control energy will be inevitably dominated by the final snapshot A M during the last leg of the system's journey from x t M −1 to x t M = x f . Thus, although it appears changing network structure is required for dramatic control advantages over static networks, the precise effects of temporality can nonetheless be understood by appealing to a single snapshot.            (2) max are the maximum eigenvalues of A 1 and A 2 , respectively. The scaling can be divided into three cases according to the sign of λ max . The Heaviside step function H(x) satisfies H(x < 0) = 0 and H(x ≥ 0) = 1. These analytical results are validated by numerical calculations, shown in Fig. 1, and the corresponding panels are given as the last column. The more general case of a temporal network with M > 2 snapshots can be found in SI Sec. C. Fig. 1c, 1f min , and λ (1) min and λ (2) min are the minimum eigenvalues of A 1 and A 2 , respectively. The scaling can be divided into three cases according to the sign of λ . Numerical calculations are also given in Fig. 2. The more general case of a temporal network with M > 2 snapshots can be found in SI Sec. C.

FIG. 1. Lower bound of the minimum energy needed for controlling temporal and static networks.
Numerical results agree with the theoretical calculations shown in Table I, where each case is divided according the maximum eigenvalue λ (2) max of the second snapshot A 2 . We set λ (2) max ≤ 0 (a). All notation is the same as that in Table I. The corresponding results for the case of more snapshots can be found in Figs. S1 and S2. All results correspond to a single representative network where N = 20, k = 6, w ij ∈ (0, 1) uniformly, with a single node randomly chosen to receive the input signal.

FIG. 2. Upper bound of the minimum energy needed for controlling temporal and static networks.
Numerical results agree with the theoretical calculations shown in Table II, where each case can be divided according the minimum eigenvalue λ (2) min of the second snapshot A 2 . We employ λ (2) min = 2, −2, and 0 to cover the cases in which A 2 is PD (a and d), NPD (b and e), and PSD (c and f), respectively. When the duration time h of each snapshot is short, the maximum energy for a temporal network is always less than that of its static counterpart (see the inset of each panel). Furthermore, as λ (1) min increases, E decreases exponentially (second row). E always decreases exponentially for temporal network when λ (2) min ≤ 0 for large h (see (a)). All notation is the same as that in Table II. The corresponding results for the case of more number of snapshots can be found in Figs. S3 and S4. All results correspond to a single representative network where N = 8, k = 4, w ij ∈ (−1, 0) uniformly, and a single node was chosen randomly to receive the input signal.
Temporal Static Temporal Static Temporal Static

VIII. SUPPLEMENTARY INFORMATION
Appendix A: Control energy for two snapshots and one driver node from x 0 = 0 to x f We denote by A(i, j) = a ij the entry at ith row and jth column in a matrix A, and let A 1 = (a ij ) N ×N and A 2 = (b ij ) N ×N , where N × N represents the size of the corresponding matrix. We assume without loss of generality that it is the c-th node that receives direct input, meaning we have B T = (0, · · · , 1, · · · , 0) where the cth entry is 1 while others are 0.
When the two snapshots of the temporal network are undirected, the corresponding dynamical matrices A 1 and A 2 are symmetric, allowing us to write A 1 = PΘP T and A 2 = QΓQ T , where As we control the temporal network from x 0 = 0 to x f , we have that the effective gramian matrix is for which we can expand the two component terms using the above eigendecompositions as which results in This allows us to analyze R 1 and R 2 in terms of the magnitude of h as follows: When h → 0, we can make the approximation e (γ k +γ l )h ≈ 1 + (γ k + γ l )h. Then we have where the three terms in the final expression obey Thus we have and by adding R 1 and R 2 we obtain As for the associated eigenvalues, we must solve the following equations This yields the approximated eigenvalues as λ = 0 (with multiplicity N − 2), Therefore, in this case, i.e., h → 0, we have

For large h
For a square matrix, the trace of the matrix is the sum of the eigenvalues. Here when h is large, we use the trace of SWS T to approximate its maximum eigenvalue, i.e., Hence we obtain that Therefore, the scaling of E for controlling temporal networks from From the numerical calculations, we have the scaling of E for controlling temporal networks from Appendix B: Control energy for two snapshots and one driver node from x 0 to x f = 0 If there are two snapshots A 1 and A 2 , and x f = 0, we can follow a similar procedure to the above to write where the individual terms can be expanded as From the following relation This allows us to analyze C 1 and C 2 according to the magnitude of h.

As h → 0
By making the approximation e −(γ k +γ l )h ≈ 1 − (γ k + γ l )h, we have Furthermore, we obtain Thus we have As for the associated eigenvalues, we must solve the following equations This yields the approximated eigenvalues as λ = 0 (with multiplicity), Therefore, in this case, i.e., h → 0, we have

For large h
When h is large, we use the trace of C to approximate its maximum eigenvalue, i.e., We know that Based on the above expressions, we have Therefore, the scaling of E for controlling temporal networks from Similarly, the scaling of E for controlling temporal networks from Appendix C: Control energy for M snapshots and one driver node When there are M snapshots, we have where there are now M terms analogous to R 1 and R 2 in the two-snapshot cases above, namely

For each snapshot, we have
N . Then we obtain For small h l , we have 1 For large h l , we have Therefore, the scaling of E for controlling temporal networks with M snapshots from x 0 = 0 to And the overall scaling of E for controlling M temporal networks from Similarly, the scaling of E for controlling temporal networks from x 0 to x f = 0 with M snapshots is obtained as And the scaling of E for controlling temporal networks from

Appendix D: Control energy for p driver nodes
Here we provide the derivation of the control energy scaling for controlling temporal networks from x 0 = 0 to x f with p driver nodes, generalizing the single driver node case shown above. Other cases with x 0 = 0 can be obtained based on the similar generalization from a single driver node case shown in Sec. B.
Assuming that there are p driver nodes with u(t) = (u 1 (t), u 2 (t), · · · , u p (t)) T , and the set of nodes receiving inputs is the set I = {i 1 , i 2 , · · · , i p }. Without loss of generality, we can relabel the network nodes so that the control inputs correspond to nodes 1, 2, · · · , p by letting i j = j for j = 1, 2, · · · , p.
Hence node j corresponds to the input u j (t), and we have I = {1, 2, · · · , p}. Then we obtain that B ii = 1 for i = 1, 2, · · · , p, with all other entries of B equal to 0. We shall first consider the control energy with two snapshots, and from there generalize to an arbitrary number of snapshots.

For two snapshots
In this case, the analogous terms R 1 (i, j) and R 2 (i, j) that appear in equations (A1) and (A2) of the effective gramian matrix are Denoting BB T = (B xy ) N ×N , we have Q ir e (γr+γn)h Q sr Q mn Q jn N k=1 p c=1 N l=1 P sk P ck P cl P ml θ k + θ l e (θ k +θ l )h − 1 , Then we analyze the maximum eigenvalue λ max of SWS T as follows.
a. As h → 0 In this case, we obtain and , , . Furthermore, we have As for the associated eigenvalues of SWS T , we must solve the following equations When h → 0, the determinant of SWS T − λI can be approximated by its first-order expression with respect to h, which yields |SWS T − λI| ≈ (2h − λ) p (−λ) N −p . Hence we have When h is large, we have .
Hence we obtain Therefore, the scaling of E for controlling temporal networks from x 0 = 0 to x f with p driver nodes is By fitting the numerical calculations, we have the scaling of E for controlling temporal networks from x 0 = 0 to x f with p driver nodes is

For an arbitrary number of snapshots
If there are M snapshots, we have for the analogous terms that contribute to the effective gramian matrix (detailed notations are given in Sec. C).
For small h l , we have 1 which leads to E ∼ h −1 for short snapshot durations.
Conversely, for large h l we have Therefore, the scaling of E for controlling M temporal networks from And the scaling of E for controlling temporal networks with M snapshots from Equivalently, the optimal control energy for static networks can be obtained by considering a temporal network with M identical snapshots, i.e., A i = A s for i = 1, 2, · · · , M . In this case we have the effective gramian matrix Thus in either view, the energy for controlling temporal networks recapitulates the known result for static networks [11]. Indeed, for controlling a static network from x 0 = 0 to x f our results indicate that the energy is bounded below by and bounded above by where δ 1 and δ N are the maximum and minimum eigenvalues of A s , respectively. Similarly, for controlling a static network from x 0 to x f = 0, the optimal energy bounds obey Taken together, we know that scaling behavior of the control energy for temporal networks reduces to the static network case as shown in [11] when all snapshots are identical.
as controlling the static network from N , and the node c receives the control input directly. Similarly, controlling from x 0 to x f = 0, we have The total energy for controlling temporal network from x 0 to x f can be viewed as the summation of the energy over each snapshot (given in Eq. (E1)), i.e.
where x i−1 and x i are the initial and final states over the snapshot A i , and h is the duration time for each snapshot.
When E(x 0 , x f ) reaches its minimum value E * (x 0 , x f ), the series of intermediate states-x(t) at times t = mh when the network structure changes (m = 1, 2, · · · , m − 1)-should satisfy for i = 1, 2, · · · , M − 1. From the above equation, we know E(x 0 , x f ) = E * (x 0 , x f ) when the following relation holds 1 is the maximum eigenvalue of the snapshot A i with i = 1, 2, 3, 4, 5. γ, δ, and λ determine the energy scaling behavior for controlling temporal network E t and static network E s . We employ λ (5) 1 = 2, −2, and 0 to represent the Not Negative, Negative, and Negative Semi definite of the last snapshot matrix A 5 , respectively. For the constant numbers, c)), and C 3 = −1/A −1 5 (c, c), where A s and A 5 correspond to the network matrix in each case. Here N = 8, k = 4, and the node receiving the input directly is node 3. For each setting of the λ (i) 1 , the numerical validation of our theoretical result is presented in Fig. S2, and here we indicate the corresponding panel.

Equation (G1) can also be written as
Maximum eigenvalues λ 2 -2 -2 2 4 2 e −8 e −4 (c) 0 -2 C 2 C 1 (g) 2 0 e −4 h −1 (k) 2 -2 2 -2 2 2 e −4 e −4 (c) -2 -2 is the minimum eigenvalue of snapshot A i with i = 1, 2, 3, 4, 5. γ, δ, and λ determine the scaling behavior for controlling temporal network E t and static network E s . We employ λ (5) 1 = 2, −2, and 0 to represent the Positive, Not Positive, and Positive Semi definite of the last snapshot matrix A 5 , respectively. For the constant numbers, c)), and C 3 = −1/A −1 5 (c, c), where A s and A 5 correspond to the network matrix in each case. Here N = 8, k = 4, and the node receiving the input directly is node 5. For each setting of the λ (i) N , the numerical validation of our theoretical result is presented in Fig. S4, and here we indicate the corresponding panel.

0)
A 5 is NSD   Table S1, and the numerical calculations are in excellent agreement with the theoretical results shown in the same table.

0)
A 5 is NSD  Table S1, and the numerical calculations are in excellent agreement with the theoretical results shown in the same table.  Table S2, and the numerical calculations are in excellent agreement with the theoretical results shown in the same table.  Table S2, and numerical calculations are in excellent agreement with the theoretical results shown in the same table.


PARALLEL IMPLEMENTATION OF THE INFORMATIVE AREAS GENERATION METHOD IN THE SPATIAL SPECTRUM DOMAIN

This paper proposes a parallel implementation of th e image informative segments extraction method. The images are segmented in the spatial spectrum do main. The median energy in each selected segment is viewed upon as an area. For purposes of time saving s, a parallel implementation of the algorithm for c alculating the areas is developed. The developed appr o ch to the parallel algorithm implementation is te sted on a high performance multicore computing system. T he experiments have shown that the parallel implementation of the method allows us to obtain a three -fold speedup, which is a good result.

Introduction
Currently computer processing of medical diagnostic images is one of the vital research tools and a way to improve efficiency of early detection of various diseases. Change of the body fluids composition is one of the information-bearing health condition areas [1 -3]. Metabolic change that occurs due to pathological conditions affects the fluid composition; there are numerous changes in the molecular composition of tissues and body fluids. Converting the fluid from one phase state to another is one of the ways to detect such changes. Crystallization is one of the most convenient methods to change the fluid phase. Crystal properties modification is caused by changed physical and chemical properties of a body fluid. The investigation of these properties is the crucial problem of crystal analysis [4]. In medicine, studied crystallograms are the structures formed by salt crystallization caused by body fluid drying.
In clinical practice the crystallogram analysis is based on their images. It is not always possible to visually identify changes in such key crystallogram parameters as predominant bar direction, bar density etc., which contribute to major pathologic signs. Quantified analysis and objectivity are among computer analysis advantages.
The information contained in the image is structurally excessive. It is known that if the parallel bars of certain direction were predominant on the original image, the bars of the same direction would dominate the Fourier transform of an original image. This property can be used to analyze crystallograms [5,6] and other images of branched structure.
The developed method based on discriminatory analysis algorithm is applied to generate the informative areas set, which is used in this paper to identify the characteristics of the initial crystallogram images.
In the article, we propose a parallel implementation of the method to speed up computations.

Informative areas generation method
In this work, the areas are derived from calculation of the total energy on the selected spectrum image ranges. Most part of the spectrum does not contain the information suitable to identify the characteristics of an original image.
If the image function and its Fourier transform F(u, v) are considered in a spatial domain, then the magnitude |F(u, v)| 2 defines an energy spectrum of the image. The energy spectrum of the image can be directly analyzed as a whole or partially.
In this work, we analyzed features derived by calculating the total energy of a selected domain of the spectrum image. The spectrum image in the domain of interest was segmented using a formula: Since the spectral image is symmetric around the center, only half of the image will be used to form up the areas [7,8].
This paper describes the method to extract the informative segments from the spectrum images (in Fig. 1 is shown as a stage of smart area analysis process). The segment informative value was estimated using the criteria of separability of discriminatory analysis algorithm.
The methods based on the discriminative analysis algorithm proved to be a good solution to form up new problemspecified areas [9]. These methods permit to improve the reliability of data classification [10 -12].
The discriminative analysis is used to eliminate the correlation between the areas and consequently to reduce the size of the areas set. The usage of this algorithm allows, on the one hand, to maintain the informativeness of the feature set for classification and, on the other hand, to reduce the number of areas to apply less complicated classification methods and to reduce the classification error value.
An individual separability criterion was calculated for each area based on equation: J=tr( (T) -1 B), where T = B + W, B is between-group scattering matrix, W is intragroup scattering matrix.
The informative areas set is further generated in the following manner: 1. The areas are ranged in the order of decreasing of individual separability criteria values. 2. The initial areas set consists of a feature with the largest criteria. Classification is carried out. 3. Then the area with the next value of separability criterion is added to the set. Classification based on the new set of areas is carried out. 4. Repeat item 3 until all areas are included into the set.
The informative set of areas is the one with classification results yielding the minimum error value. The classification error that defines the number of cases with the classifier acquiring incorrect value is calculated from the equation: ( / ) 100% m n ε = i , -where m is the number of classification errors, n is the total number of images tested.
In this work, with a view of accumulating statistics we utilized crystallographic images of a lachrymal fluid. The images were divided in two classes based on a visual analysis. The training set was composed of 200 samples of 256×256 pixels, containing 100 images of each class. The images under testing had similar parameters, with the samples containing 100 images in each class. Based on this data, image classification was done.

Parallel implementation of the method to generate informative areas
The analysis of the algorithm structure showed that the first stage of areas computing has the maximum computational complexity.
As mentioned above most of the algorithm run time is spent on computing the areas. This stage includes preprocessing of the learning sample, generation of spatial spectrum for each learning image and calculation of the area values. The number of calculations can be reduced if one take into account that the spatial spectrum image is symmetrical relative to its center, and for this reason the areas calculation may and must be performed only on the half of the spatial spectrum image. In order to speed up the algorithm run time this paper will apply task division. This paper will not use an MPI technology but will apply task distribution by threads, that is why the way of splitting the image into tasks is not important. In case of sample pre-processing and spatial spectrum image generation a single image is sent to each thread. Then at the areas calculation stage each thread reveives a separate element -an image segment calculated on the basis of the proposed segmentation. The next task is sent to the thread as soon as it completes handling of the element.
Application of this segmentation pattern allows substantial run-time saving at the first stage of the algorithm. During this resource-intensive stage, such method of tasks segmentation by threads has permitted to achieve the three-fold speedup when four threads were used. The curve in Fig. 1 shows the relation between speedup and the number of areas.
To evaluate the work developed by the implementation performed experiments were performed on personal computer running Windows 7 operating system with Intel Core i5 4460 processor.

Classification results
After classification for all possible segmentations, areas were selected using the developed area selection algorithm. Fig. 2 shows the relation between classification error and the number of areas taken in the descending order of their separability criterion in case of classification with 2-nearest neighbors methods to split into 4 sectors and 8 rings.

Fig. 2. Relation between classification error and the number of areas
Tables 1 show classification error value after selecting the informative area set. The error value selection to be included into the final table is shown above.

Conclusion
This paper presented the method to generate a set of informative local areas of a spatial spectrum in order to classify medical crystallogram images, and an option of parallel implementation of such method. In addition, the research included experimental testing of the developed software implementation, which demonstrated that parallel algorithm implementation provided almost threefold speedup at the areas calculation stage. The next step is to implement parallel computing at the smart area analysis stage in order to improve the informative area set calculation speed.


extfieldsofstudy:
Computer Science
Mathematics
Medicine
Sociology

text:
Self-organization of meaning and the reflexive communication of information

Following a suggestion from Warren Weaver, we extend the Shannon model of communication piecemeal into a complex systems model in which communication is differentiated both vertically and horizontally. This model enables us to bridge the divide between Niklas Luhmann’s theory of the self-organization of meaning in communications and empirical research using information theory. First, we distinguish between communication relations and correlations among patterns of relations. The correlations span a vector space in which relations are positioned and can be provided with meaning. Second, positions provide reflexive perspectives. Whereas the different meanings are integrated locally, each instantiation opens global perspectives – ‘horizons of meaning’ – along eigenvectors of the communication matrix. These next-order codifications of meaning can be expected to generate redundancies when interacting in instantiations. Increases in redundancy indicate new options and can be measured as local reduction of prevailing uncertainty (in bits). The systemic generation of new options can be considered as a hallmark of the knowledge-based economy.

Introduction
In his contribution to Shannon and Weaver's book, The Mathematical Theory of Communication, Warren Weaver stated (1949: 27) that '[t]he concept of information developed in this theory at first seems disappointing and bizarre -disappointing because it has nothing to do with meaning …'. However, the author added that Shannon's analysis 'has so penetratingly cleared the air that one is now, perhaps for the first time, ready for a real theory of meaning'. But how can one relate a theory of meaning to Shannon's information theory (cf. Bar-Hillel & Carnap, 1953)? More recently, Niklas Luhmann (1995c) argued that meaning (Sinn) self-organizes in terms of communications among human beings. From this perspective, meaning is generated in interactions among communications. As Luhmann (1996: 261) formulated it: 'My argument is: it is not human beings who can communicate, rather, only communication can communicate.' However Luhmann's theory has remained far from operationalization and measurement.
Following Bateson's (1972: 315) alternative definition of information as 'a difference which makes a difference ' (cf. MacKay, 1969), Luhmann ( : 102ff, 1995c: 67f) considered information as implying a selection: a difference can only make a difference for a system of reference that selects this difference from among other possible differences. Others have also defined information with reference to a receiving system (e.g. an observer). Varela (1979: 266) even argued that, since the word 'information' is derived from 'in-formare', the semantics call for the specification of a system of reference to be informed. 'Information', however, is then considered a substantive concept that varies with the system of reference instead of a formal measure of the uncertainty prevailing in a distribution. Kauffman et al. (2008: 28), for example, defined information as 'natural selection assembling the very constraints on the release of energy that then constitutes work and the propagation of organization'. In summary, using Bateson's alternative definition of information, the meaning of 'information' becomes dependent on the theoretical context. Using the same word (information) for different concepts has led to considerable confusion in the literature.
In an assessment of this confusion, Hayles (1990: 59f) compared the discussion with asking whether a glass is half empty or half full. As she noted, confusion can be avoided by using the words 'uncertainty' or 'probabilistic entropy' when Shannon-type information is meant. In our opinion, the advantage of measuring uncertainty -and redundancies, as we shall argue -in bits of information cannot be underestimated, since the operationalization and the measurement provide avenues to hypothesis testing and thus control of the theorizing (Theil, 1972). Note that uncertainty cannot be specified in terms of Bateson's definition, but his 'a difference which makes a difference' can be operationalized and measured in terms of (potentially negative) bits of information (Brillouin, 1962;Von Foerster, 1960).
Can Luhmann's theory about interacting communications and the self-organization of meaning also be made compatible with Shannon's information theory? Is it possible to specify how information and meaning are related? In this study, we aim to contribute to bridging this gap between a focus on meaning versus uncertainty processing by decomposing the problem using Herbert Simon's (1973a) model of complex systems that are both vertically and horizontally differentiated, as follows: 1. In the vertical dimension, we follow Luhmann's (e.g. 1975Luhmann's (e.g. , 2000 distinction between (i) interactions among communications providing variation; (ii) the organization of meanings in historical instantiations; and (iii) the self-organization of reflexive meaning generating a next level of 'horizons of meaning' as global systems of reference (Husserl, 1960(Husserl, [1929; cf. Luhmann, 1995b). Despite this inspiration from Luhmann, however, we also deviate from his framework and argue that the construction in terms of layers is bottom-up from the (probabilistic) informational level; but the emerging system's levels can be expected to take over control in terms of codified intentionalities and expectations. In other words, the operation in layers can also be described in terms of variation and selection mechanisms. For example, a scholarly communication (e.g. a manuscript) can be expected to contain a knowledge claim. Knowledge claims provide in this case the historical variation. When the manuscript is submitted, an editorial process is instantiated in which referee comments, editorial judgments, etc. are combined. The referees, however, are expected to judge the manuscript in terms according with the standards of the field invoking the codes of communication that can be expected to control the process. The instantiation requires a reflexive reorganization of the codes.
One can consider the constructs from two sides, the first being the constructing agency (in history) and the second being the resulting constructs themselves, which can only be entertained reflexively (Giddens, 1979). 1 Because of this reflexive status, the processing (e.g. sharing of meaning) is volatile and cannot be observed directly; but the operations can be specified. On the basis of this specification, one is able to identify and measure the 'footprints' of the self-organizing dynamics in historical instantiations. In other words, intentions and intentional systems cannot be found as observables in res extensa -or 'matter' -but can only be hypothesized reflexively in res cogitans -'thought' (Husserl, 1960(Husserl, [1929; Luhmann, 1990). Entertaining such hypotheses can enrich our expectations by providing frames for inferences about observations. The consequent possibility of an inversion in the order of control from the material conditions to systems of expectations enables reflexive agents -human beings -to operate infra-reflexively across (vertical) levels and among (horizontal) compartments by changing their perspectives on the complexity (Latour, 1988;Pickering, 1995). 2. In the horizontal direction, we follow Talcott Parsons' (1968) proposal to consider the functional differentiation and symbolic generalization of the codes of communication as drivers of the increasing complexity in cultural evolution.
Recalling another intuition of Simon (1973a: 19ff), one can expect an alphabet of these codes; for example, power, love, truth, law, art, etc. Because of the various codes operating, the same communication can mean something quite different in terms of its affective value, its truth value, or how power is reproduced in communications (Künzler, 1987;Luhmann, 1974Luhmann, , 2013; Parsons, 1963aParsons, , 1963bParsons, , 1968). The complexity is increased because the codes are also recombined in their instantiations (Hoffmeyer & Emmeche, 1991). A dynamics of differentiation among the codes versus integration in instantiations is thus to be specified. Based on decoding and recoding, for example, differently codified expectations about markets and technologies can be recombined into new technological options (Arthur, 2009;Cowan & Foray, 1997). A technological evolution can thus be generated as a retention mechanism of the cultural evolution of possible expectations (Dubois, 2003).

The generation of meaning from (Shannon-type) information
How can the processing of meaning be conceptualized by elaborating on Shannon's theory despite the author's explicit statement that the 'semantic aspects of communication are irrelevant to the engineering problem' (Shannon, 1948: 3)? As a first step in the specification of the relevance of Shannon's engineering model for developing a theory of meaning, Weaver (1949: 26) proposed two 'minor additions' to Shannon's well-known diagram of a communication channel (Figure 1), as follows: One can imagine, as an addition to the diagram, another box labeled 'Semantic Receiver' interposed between the engineering receiver (which changes signals to messages) and the destination. This semantic receiver subjects the message to a second decoding, the demand on this one being that it must match the statistical semantic characteristics of the message to the statistical semantic capacities of the totality of receivers, or of that subset of receivers which constitute the audience one wishes to affect.
Similarly one can imagine another box in the diagram which, inserted between the information source and the transmitter, would be labeled 'Semantic Noise', the box previously labeled as simply 'noise' now being labeled 'Engineering Noise'. From this source is imposed into the signal the perturbations or distortions of meaning which are not intended by the source but which inescapably affect the destination. And the problem of semantic decoding must take this semantic noise into account.
Since the 'semantic receiver' recodes the information in the messages (received from the 'engineering receiver', who only changes signals into messages) while having to assume the possibility of 'semantic noise', a semantic relationship between the two new boxes can also be envisaged. Given Shannon's framework, however, this relation cannot be considered as another information transfer -since semantics are defined as external to Shannon's engineering model.
Semantics are not based on specific communications, but on relations among patterns of relations or, in other words, correlations. Two competing firms, for example, may have highly correlating patterns of relations with clients, but no relation with each other (Burt, 1982). The correlations among the distributions span a vector space in a topology different from the network space of relations (Appendix). 2 Two synonyms, for example, can have the same position (and meaning) in the vector space, yet never co-occur in a single sentence as a relation. Meanings can be shared also without a direct relation.
In the case of a single relation, the relational distance is not different from the correlational one; but in the case of relations involving three (or more) agents, the distances in  Weaver's (1949) 'minor' additions penciled into Shannon's (1948) original diagram.
the vector space are different from the Euclidean distances in the network space. Simmel (1902) already noted that the transition from a group of two to three is qualitative. In a triplet, the instantiation of one or the other relation can make a difference for the further development of the triadic system of relations.
A system of relations can be considered as a semantic domain (Maturana, 1978). In other words, the sender and receiver are related in the graph of Figure 1, while they are correlated in terms of not necessarily instantiated relations in the background. The structure of correlations provides a latent background that provides meaning to the information exchanges in relations. 3 The correlations are based on the same information, but the representation in the vector space is different from the graph in the network space of observable relations. In other words, meaning is not added to the information, but the same information is delineated differently and considered from a different perspective (including absent relations, i.e. zeros in the distribution). As against Shannon-type information which flows linearly from the sender to the receiver, one can expect meanings to loop, and thereby, to develop next-order dimensionalities (Krippendorff, 2009a(Krippendorff, , 2009bLeydesdorff, 2010).

The third and fourth dimensions of the probabilistic entropy
A matrix of communications is shaped when one adds a second dimension (of codes) to the single vector of communication in Figure 1. A matrix can also be considered as a two-dimensional probability distribution -different from the one-dimensional probability distribution in a vector. When the communication matrix -of information processing and meaning processing -is repeated over time, one obtains a three-dimensional array because time is added as a third dimension ( Figure 2a). In such a three-dimensional array, the development of information can also be considered in terms of trajectories; the uncertainty is then organized historically (over time). A four-dimensional array or hypercube of information is more difficult to imagine or represent graphically: unlike a (threedimensional) trajectory, a four-dimensional array can contain a next-order regime that feeds back on its historical development along trajectories (Dosi, 1982).
One can consider the next-order regime as having one more degree of freedom, which allows it to select among the possible trajectories in three dimensions as representations of its past ( Figure 2b). This additional selection implies a reflection by the system. The reflection is performative in the present and can therefore be considered as the selforganization of an adaptive system. In the four-dimensional system, one representation of its history in three dimensions can be acknowledged (weighted) more than another.
We can consider ourselves as psychologies with the reflexive capacity to reconstruct the possible representations of our history. Luhmann (e.g. 1986b) suggested modeling the social system of communications as a system without psychological consciousness, but with an equal level of complexity. Whereas a psychological system is centered on the individuum and will therefore tend to integration (Haken & Portugali, 2014), the communication system is distributed (as a 'dividuum'; Luhmann, 1984: 625;cf. Nietzsche, 1967cf. Nietzsche, [1878: 76) and has the option of exploiting the additional degree of freedom for differentiation. Different from a high culture, the modern society is based on prevailing differentiation among codes of communication, so that a set of juxtaposed coordination mechanisms can be used. The different coordination mechanisms are partially integrated when the systems are instantiated in terms of historical organization and action.
In terms of evolution theory, the coordination mechanisms (e.g. the market) can also be considered as selection mechanisms that operate with different criteria. The recursion of selection at the structural level leads to second-order variation: selections can be selected for stabilization (in history), and some stabilizations can be selected for globalization (at the regime level). For subsequent selections, however, the historical origin of a variation is not always relevant. Thus the system of expectations may continuously loop into itself at different levels and from different perspectives. Weaver (1949: 24) suggested taking Shannon's original diagram as a representation of 'level A', which can be complemented with more levels (B and C) that represent how meaning is conveyed at level B, and how and why the received meaning can affect behavior (at level C). Elaborating on Shannon's model and Weaver's addition as depicted in Figure 1, we propose Figure 3 as a scheme for levels B and C.

Levels B and C in the Shannon diagram
We specified above that the relation between the semantic receiver and semantic noise is based on correlations among sets of relations at level A. In the vector space (level B), meanings can be shared but not communicated (because otherwise one operates at level A). The use of language facilitates, supports and potentially reinforces the options for sharing meaning. Natural languages (at level B), however, can be considered as the yet-undifferentiated and therefore common medium of communication. Codes of communication are used at the symbolic level C for regulating the use of language. The codes enable us, among other things, 4 to shortcut the communication; for example, by paying the market price of a good instead of negotiating this price using language. In our opinion, the codes of communication are thus candidates for Weaver's level C: the codes and their combinations enable us to make the communications far more precise and efficient than is possible in natural languages. Parsons (1968: 440) provided a sociological appreciation of the operations at level C as follows: At the cultural level [language] is clearly the fundamental matrix of the whole system of media. Large-scale social systems, however, contain more specialized media (if you will, specialized 'languages'), such as money, power, and influence (see Parsons 1963a,b). Such media, like language, control behavior in the processes of interaction.
In addition to symbolic, Parsons (1963aParsons ( , 1963bParsons ( , 1968) characterized these media as 'generalized', with a reference to Mead's (1934: 154ff) 'generalized other'. Luhmann (1974) further argued that 'symbolically generalized media of communication' would have to be binary -like true and false in logics -in order to be binding (Luhmann, : 316f, 1995acf. Künzler, 1987: 329ff). In our opinion, the codes can be more complex than one-dimensional and binary (Hoffmeyer & Emmeche, 1991). For example, Herbert Simon (1973a) argued in favor of truth-finding and puzzle-solving as combined 'logics' in scientific discovery. However this operationalization has also remained chiefly a philosophical appreciation of the evolutionary process (cf. Popper, 1959Popper, [1935) more than a proposal for empirical operationalization (cf. Newell & Simon, 1972).
In our opinion, the sciences evolve as systems of expectations rationalized by arguments in discourses; after a further development, the criteria may also have changed Source: adapted from Shannon (1948). (Fujigaki, 1998;Kuhn, 1962). Using the conceptualization of the codes as the latent dimensions (principal components or 'eigenvectors'; 5 Von Foerster, 1960;cf. Von Glasersfeld, 2008: 64 n4) of the communication matrix, one can appreciate the uncertain and evolving character of these codes of communications. From this perspective, the designation of these structures remains a historical appreciation.
For example, in the case of the science system, Luhmann (1990) argued for true/not-true as the code that provides a binary criterion for quality control in scholarly discourse. However, in the empirical sciences, the truth of statements is not unambiguous: some statements can be more true or less false than others. Since the sciences develop as discursive knowledge, uncertainty is always present. In a study of the debates about oxidative phosphorylation -which led to the Nobel Prize in Chemistry for Peter Mitchell in 1978 - Gilbert & Mulkay (1984) found that statements were relabeled using different repertoires when they were considered true or erroneous from the perspective of hindsight.
Codes can also be nested. For example, different specialties may operate with different codes while sharing some general criteria for the quality control of scholarly communications. Similarly in economic transactions a variety of payment methods can be distinguished under the umbrella of an economic logic that differs from a scholarly or normative one (Boudon, 1979;Bourdieu, 1976).
In his last book, Pierre Bourdieu (2004: 83) precisely formulated a reflection on the empirical study of the sciences, as follows: Each field (discipline) is the site of a specific legality (a nomos), a product of history, which is embodied in the objective regularities of the functioning of the field and, more precisely, in the mechanisms governing the circulation of information, in the logic of the allocation of rewards, etc., and in the scientific habitus produced by the field, which are the condition of the functioning of the field. … What are called epistemic criteria are the formalization of the 'rules of the game' that have to be observed in the field, that is, of the sociological rules of interactions within the field, in particular, rules of argumentation or norms of communication. Argumentation is a collective process performed before an audience and subject to rules. Bourdieu (2004: 78) calls this a 'Kantian' -that is, transcendental -transition from 'objectivity' to 'intersubjectivity' as the carrying ground of scientific inferences. However, the philosopher most associated with this transition is Edmund Husserl, who criticized the increasingly empiristic self-understanding of the modern (European) sciences (Husserl, 1962(Husserl, [1935). According to Husserl (1960Husserl ( [1929: 155), the possibility to communicate expectations intersubjectively grounds the empirical sciences 'in a concrete theory of science'. One tests expectations entertained at the supraindividual level (in discourses) against observations, and the observations can update the expectations since they can function as arguments.
We shall take from Husserl the proposition that the self-organizing codes of the communication at level C are not material, but belong to our reality as structures of expectations or res cogitans. Popper (1972) denoted this domain as 'World 3', but neither he nor Husserl specified the evolutionary dynamics of expectations in terms of communications (Luhmann, 1986a). We submit that res cogitans can be expected to develop in terms of redundancies instead of probabilistic entropy, unlike the material world (res extensa), where the Second Law of thermodynamics prevails.
Language and the symbolic media of communication enable us to multiply meanings as options at a speed much faster than can historically be realized. The codes of communication provide us with horizons of other possible meanings. The different codes can be recombined and reconstructed in translations among differently coded meanings. We shall argue that at level B meanings are instantiated in specific combinations of codes, while at level C the codes evolve in response to the historical integrations in the instantiations.

The transformation of hitherto 'impossible' options into technologically feasible ones
Whereas historical developments unfold with the arrow of time -and are necessarily related to the generation of entropy -expectations enable us to use possible future states in the present, i.e. against the arrow of time. The dynamics of expectations therefore are very different from organizational dynamics. Under specifiable conditions, the interactions among differently coded expectations can generate redundancy (that is, negative entropy). Redundancy enriches a system with new options that are available for realization.
The redundancy R is defined in information theory as the fraction of the capacity of a communication channel that is not used. In formula format: As is well known, Shannon's (1948) probabilistic entropy (H) is coupled to Gibbs' formula for thermodynamic entropy S k H B = * . In Gibbs' equation, k B is the Boltzmann constant that provides the dimensionality Joule/Kelvin to S, while H is dimensionless and can be measured in bits of information (or any other base of the logarithm) given a probability distribution (containing uncertainty). The Second Law of thermodynamics states that entropy increases with each operation, and Shannon-type information is accordingly always positive. Brooks and Wiley (1986) noted that, in the case of an evolving (e.g. biological) system, not only the observed (probabilistic) entropy (H obs ) of the system increases, but also the maximum entropy (H max = ln N) -that is, the system's capacity -increases as the total number of possible options (states) N also increases. The difference between the maximum entropy and the realized entropy is provided by the options that are available but have not yet been used. From the perspective of information theory, these surplus options are redundant. Using Brooks and Wiley's (1986: 43) illustration in Figure 4a, we added green to the redundancy as part of the evolving entropy. Redundancy provides a measure of the options that were not realized by the system but could have been realized. Kauffman (2000), for example, called these possible realizations 'adjacent'.
Above the green area, Brooks and Wiley (1986: 43) added the legend 'impossible' (Figure 4a). In Figure 4b, we have added the domain 'technologically made feasible' to this latter area in order to introduce how the generation of new options (and hence increased redundancy) can be enhanced by a model of cultural evolution that includes levels B and C. An intentional system operates by adding new options (redundancy) without necessarily realizing them.  The codes regulate the generation of redundancies at interfaces from above, whereas Shannon entropy is continuously generated in the historical process from below. The latter process is linear, whereas expectations can circulate before being organized in realizations. Redundancy is generated when two (or more) perspectives on the same information are operating at an interface, as in the case of introducing a new technology in a market or when writing a report based on scholarly arguments for a government agency. In such cases, one needs text that can be read using the various codes involved (Fujigaki & Leydesdorff, 2000). The redundancy represented by the green surfaces of Figure 4b is generated by the recombination of sufficiently different expectations. Let us try to specify this process in information-theoretical terms.

Mutual redundancy between two differently coded systems
In Figure 5, the overlap between uncertainties in two variables x 1 and x 2 is depicted as two sets. The mutual information or transmission (T 12 ) is then defined in information theory as follows: Note that the addition of entropies accords with the rules of set theory. Alternatively one can consider the overlap as a redundancy: the same information is then appreciated twice. In addition to H 1 and H 2 , the overlap contains redundancy as a surplus of information, as follows: The mutual redundancy R 12 at the interface between the two sets can now be found by using Y 12 instead of H 12 in Equation 2, as follows: Since T 12 is necessarily positive (Shannon, 1948: 53), it follows from Equation 4 that R 12 is always negative and therefore by definition a redundancy. This reduction of the uncertainty can be measured in bits of information with a negative sign. In other words, the redundancy cannot be a Shannon-type information since the latter information is necessarily positive (Krippendorff, 2009b). Using mutual redundancy, one no longer measures a historical process -generating uncertainty -but a process in the realm of expectations: future states are represented in the present (Dubois, 1998).

Redundancy in three and four dimensions
For the three-dimensional case and using Figure 6, one can define, in addition to the twodimensional values of Υ (in Equation 3), a three-dimensional value including the redundancies in the respective overlaps as follows: Using information theory, however, one would make the following corrections for double counting in the overlaps (by subtracting; see Figure 6):

H H H H T T T T
It follows that the difference between the uncertainty in the historical system (H 123 in Eq. 6) and the system of expectations (Υ 123 in Eq. 5) is: Furthermore the mutual information in three dimensions can be derived in information theory using the Shannon formulas (e.g. Abramson, 1963: 129;McGill, 1954;Yeung, 2008) In the three-dimensional case, the mutual redundancy is thus equal to the mutual information in three dimensions. Furthermore Leydesdorff and Ivanova (2014: 392) show that, in the case of four dimensions, R 1234 = -T 1234 . The sign of the mutual redundancy alternates with the number of dimensions. This corrects for the otherwise inexplicable sign changes in the mutual information with increasing dimensionality. This sign change of the mutual information with dimensions is a well-known problem in information theory, but beyond the scope of the present study. 6 In other words, mutual redundancy is a consistent measure of negative entropy, while mutual information is not, because of its sign changes with the dimensionality. We prove this claim in the next section by generalizing the formulation.

Generalization
Equation 8 The second bracket in Equation 10  The terms in the first bracket of Equation 10 are strictly positive. As noted, the sign of the resulting value of T 123 depends on the empirical configuration (as indicated by the two configurations in Figure 6). It follows (inductively) that, for any given dimension n, one can formulate combinations of mutual information corresponding to The left-bracketed term of Equation 14 is necessarily negative entropy (because of the subadditivity of the entropy), while the configuration of the remaining mutual information relations contributes a second term on the right, which is positive (see the set of Equations 11 above). In other words, we model here the generation of redundancy on the one side versus the historical process of uncertainty generation in relating on the other as an empirical balance in a system that operates with more than two codes (e.g. alphabets; Abramson, 1963: 127ff). When the resulting R is negative, self-organization prevails over organization in the configuration under study, whereas a positive R indicates conversely a predominance of organization over self-organization as two different subdynamics.

Clockwise and anti-clockwise rotations
When the relation between two subdynamics (or agents) is extended with a third, the third may feed back or feed forward on the communication relation between the two, and thus a system is shaped (Sun & Negishi, 2010). This principle is known in social network analysis as 'triadic closure' (Bianconi et al., 2014;De Nooy & Leydesdorff, 2015). Triadic closure can be considered as the basic mechanism of systems formation. When three selection environments operate on variation in the interactions among them, the communication can proliferate auto-catalytically using each third mechanism as a feedback on or feed-forward to bi-lateral relations. At a next moment, the cycling itself may take control as a vortex (Ivanova & Leydesdorff, 2014b). Ulanowicz (2009Ulanowicz ( : 1888 depicted this possibility of auto-catalysis as shown in Figure 7. A second cycle with the reverse order of the operations is equally possible; the two cycles can be modeled as two vectors with three dimensions (A, B and C), and this system can then be simulated in terms of rotations of the two vectors. One vector can be understood as corresponding to the tendency of historical realization and the other to self-organization. Using simulations, Ivanova and Leydesdorff (2014a) showed that the operation of these two three-dimensional vectors upon each other generates a value of R depending on the configuration. As we showed above, R can also be measured.
The one sign of R can be associated with clockwise and the other with anti-clockwise rotation, whereas the values of the two terms in Equation 14 measure the relative weights of the two rotations. In other words, mutual redundancy indicates the size of the footprint of the self-organizing fluxes of communication on the historical organization in the instantiation. The cycles can be vicious or virtuous in the sense of providing opportunities or exploiting existing ones.

Summary and conclusions
We have extended Shannon's model of communication (at level A) with two levels (B and C), which change the linear model into an evolutionary one because feedback and feed-forward loops are possible among the levels. At level A, information is transmitted; at level B, information is organized and thus made meaningful in a vector-space. Reflexivity reveals that this vector space is constructed and therefore a potential subject of reconstruction: the possibility of reconstruction opens horizons of meaning (level C). These horizons can be expected to evolve along the eigenvectors of the communication matrix in different directions. Whereas the common language at level B tends to integration (into organization), horizontal differentiation among the codes at level C increases the communication capacity of the system.
Codes of communication are no longer actor-attributes, but operate on the communications reflexively -that is, by co-constructing the meaning of the communication . In other theoretical contexts, one can also consider the codes as virtual coordination mechanisms (e.g. Giddens' structuration theory) or as selection mechanisms (e.g. Dosi, 1982;Nelson & Winter, 1982). As Andersen (1992: 14) noted, specification of 'What is evolving?' becomes a relevant question when selection is no longer given by nature (cf. Boulding, 1978: 33). The question arises: Under what conditions can the different selection mechanisms be expected to co-evolve and lead to new options for realizing innovations?
We showed that redundancies could be generated at interfaces among systems as sets of relations that are structured by codes, whereas in historical relations (instantiations) variety is generated. Biological evolution theory assumes variation as a driver and selection to be naturally given, while cultural evolution is driven by individuals and groups who make conscious decisions on the basis of potentially different criteria (Newell & Simon, 1972;Petersen et al., 2016).
Note that the interactions among the codes in the instantiations generate the redundancies. In Luhmann's theory, these interactions are held to be impossible: the binary codes close the autopoiesis of systems and subsystems operationally. Whereas biological systems gain in complexity by closing themselves operationally, systems of expectations can disturb one another 'infra-reflexively' (Latour, 1988: 169ff). Cultural evolution can therefore be much faster than biological evolution. The 'avoidance of redundancy' is an objective in Luhmann's model of autopoiesis and functional differentiation (e.g. 2013[1997]: 98), whereas redundancy generation is considered crucial for the advancement of society in our model. Redundancy provides new options for technological development that are not yet realized but can be envisaged.
In addition to vertical differentiation, the assumption of horizontal differentiation is needed for understanding the evolving complexity. The differentiation and the ensuing pluriformity of the competing coordination mechanisms break the structural formations of autopoietic systems so that the established meanings can be interrupted by other possible meanings. The generation of redundancy can thus enter the historical instantiations and, under the condition of self-reinforcing loops, can thereby tip the balance toward the prevalence of evolutionary self-organization over historical organization. We have shown how the trade-off between historical organization and self-organization over time can be traced by the measurement of mutual redundancies.
When three or more selection mechanisms operate, auto-catalysis is an option, and options can then be generated at an increasing pace. Thus horizontal differentiation is a necessary component of self-organization in the vertical dimension. The warp and the woof of organization of and self-organization are not harmoniously integrated, as in textiles, but differentiated and disturbing of one another. The layers are not hierarchical, but operate in parallel. These horizontal and vertical dynamics lead to a fractal manifold in different directions. Through breakages (and hence puzzles) new options can be generated.
The generation of redundancy proceeds in a domain of expectations about options that do not (yet) exist, but that one can reflexively entertain. By turning away from an objectivistic self-understanding of the sciences, we find room for a general theory of meaning and knowledge-generation, which we have depicted as an extension of Shannon's theory (Figure 3). Whereas Shannon felt the need to explicitly deny this extension, Weaver understood it as the theory's proper intension.
The Pearson correlation between these two distributions is r = 0.167 (n.s.). A natural measure of similarity between vectors is the cosine of the angle between them; in this case, cos(A,B) = 0.667. Figure A1 represents the relations in a three-dimensional Euclidean space of, for example, university-industry-government relations.
The vector space can be multi-dimensional; the dimensionality is determined by the number of vectors. Eigenvectors can be considered as vectors pointing to the centroids of clusters in a space reducing the dimensionality of the vector space (using, e.g. factor analysis or multi-dimensional scaling; Schiffman et al., 1981). See also note 5 above.