Identifying Business Sectors from Stock Price Fluctuations

Firms having similar business activities are correlated. We analyze two different cross-correlation matrices C constructed from (i) 30-min price fluctuations of 1000 US stocks for the 2-year period 1994-95 and (ii) 1-day price fluctuations of 422 US stocks for the 35-year period 1962-96. We find that the eigenvectors of C corresponding to the largest eigenvalues allow us to partition the set of all stocks into distinct subsets. These subsets are similar to conventionally-identified business sectors, and are stable for extended periods of time. Using a set of coupled stochastic differential equations, we argue how correlations between stocks might arise. Finally, we demonstrate that the sectors we identify are useful for the practical goal of finding an investment which earns a given return without exposure to unnecessary risk.

The internal structure of a complex system manifests itself in correlations among its constituents. In physical systems, one relates correlations to basic interactions, but for the stock market problem [1], the underlying 'interactions' are not known. Suppose that the change of stock prices can be visualized by the motion of point particles. Correlated particle motion can be pictured as "strings" connecting pairs of particles. Given only the records of the particle positions at equal time intervals, how can we identify the strings without 'seeing' them? One approach is to first calculate the cross-correlation matrix C whose elements C ij are the correlation-coefficients between the velocities of two particles i and j. The eigenvectors of C convey information about the collective modes of the system.
What is the analog of the cross-correlation matrix C for the stock market problem? We define the crosscorrelation matrix C with elements C ij ≡ [ G i G j − G i G j ]/σ i σ j , where σ i is the standard deviation of price fluctuations G i (t) ≡ ln S i (t + ∆t) − ln S i (t) (returns), S i (t) denotes the price of stock i = 1, . . . , N , and . . . denotes a time average over the period studied. To investigate correlations on different time scales, we analyze (i) 30-min returns of N = 1000 largest stocks for the two-year period 1994-95 and (ii) daily returns of N = 422 stocks for the 35-year period 1962-96 [2]. We first diagonalize C and rank-order its eigenvalues λ k such that λ k+1 > λ k ; the corresponding eigenvectors are denoted u k . Next, we analyze the components of those deviating eigenvectors whose eigenvalues are larger than the upper bound for uncorrelated time series [3,4]. A direct examination of these eigenvectors, however, does not yield a straightforward interpretation of their economic relevance. To interpret their meaning, we note that the largest eigenvalue is an order of magnitude larger than the others, which constrains the remaining N − 1 eigenvalues since Tr C = N . Thus, in order to analyze the contents of the deviating eigenvectors, we first remove the effect of the largest eigenvalue [5].
To analyze the information contained in the eigenvectors u k , we partition the 1000 stocks into groups labeled ℓ = 1 . . . , 75 (comprising N ℓ stocks each) according to the first two digits of their Standard Industrial Classification (SIC) code, which classifies major industry groups. We define a projection matrix P, with elements P ℓi = 1/N ℓ if stock i belongs to group ℓ and P ℓi = 0 otherwise. For each deviating eigenvector u k , we compute the contribution X k ℓ ≡ N i=1 P k ℓi [u k i ] 2 of each industry group ℓ. The above procedure of computing X k ℓ is analogous to the analysis of wave functions in disordered systems, where one calculates the probability of finding a particle in a given region. Figure 1 shows X k ℓ for ten largest eigenvectors after excluding the influence of the largest eigenvalue. The contribution X 999 ℓ shows several industries. We examine the significant contributors and find mainly stocks with large market capitalization [ Fig. 2]. We analyze X k ℓ for the remainder of the deviating eigenvectors and find a significant 'peak' at distinct values of the SIC codesuggesting that these eigenvectors correspond to distinct industry groups [6].
One deviating eigenvector u 995 displays large values of X k ℓ for firms belonging to the heavy construction industry and telecommunications industry. In addition, an examination of these firms shows significant business activity in Latin America. Another case corresponds to eigenvectors u 996 and u 997 , both of which contain a mixture of stocks of gold-mining firms and banking firms. We find that these two sectors separate when we compute the symmetric and antisymmetric combinations 1/ √ 2( u 996 ± u 997 ). The remainder of the deviating eigenvectors display technology, metal mining, banking, petroleum refining, auto manufacturing, drug manufacturing, and paper manufacturing firms [ Fig. 1].
We next focus on the interpretation of the largest eigenvalue λ 1000 . Using the eigenvector u 1000 , we construct a time series We then compare G 1000 (t) with the returns G SP (t) of the S&P 500 index, a benchmark for gauging the performance of entire US stock market. Regressing G 1000 (t) against G SP (t) shows a scatter around a linear fit with slope 0.85 ± 0.09 [ Fig. 3]. Thus, we interpret the eigenvector u 1000 as the influence of the entire market, that is common for all stocks [3,4].
Next, we examine whether the eigenvectors u k corresponding to business sectors remain stable in time. Partitioning the year 1994 into two 6-month periods, A and B, we calculate the corresponding eigenvectors u A and u B of the cross-correlation matrices and quantify the time stability by calculating the magnitude of the scalar products O ij ≡ | u i A u j B | for the 20 largest eigenvalues. Perfect time stability would mean O ij = δ ij . For i = 1000, we find O ii = 0.93 -indicating almost perfect stability. We find that O ii decreases as i decreases from 1000 [ Fig. 4]. Extending this analysis to daily returns using database (ii) shows that the eigenvectors corresponding to the largest 3 eigenvalues are stable for as many as 10 years.
How can we explain correlations that are stable in time? In physical systems, one starts from the interactions between the constituents, and then relates interactions to correlated "modes" of the system. In economic systems, we ask if "interactions" give rise to the correlated behavior. Interactions can arise when two companies are doing business together, or compete for the same market. To study if the correlations can be explained through interactions [7], we model stock price dynamics by a differential equation [8,9], which describes the 'instantaneous" returns g i (t) = d dt ln S i (t) as a random walk with interactions J ij Here, ξ i (t) are Gaussian random variables with correlation function ξ i (t)ξ j (t ′ ) = δ ij τ i δ(t − t ′ ), and τ i are relaxation times of the g i (t)g i (t+ T ) correlation function. The return G i at a finite time interval ∆t is given by the integral of g i over ∆t.
Calculating time-dependent correlation functions for the g i , we find that correlations caused by interactions are accompanied by a phenomenon analogous to "critical slowing down". The market time series G 1000 (t) -as well as time series constructed similarly for other deviating eigenvectors -have considerably larger correlation times than a time series constructed out of a random eigenvector, consistent with the hypothesis that correlations between firms are caused by interactions.
The eigenvectors that we interpret as defining business sectors also have relevance to the practical goal of finding an investment which earns a given return without exposure to unnecessary risk ("optimal portfolio"). Risk can be reduced by diversification of investment into indepen-dently fluctuating groups of stocks, such as the mutually uncorrelated business sectors that we find. Since the sectors (eigenvectors) are stable in time, we expect the ratio of risk to return of the portfolios constructed from them to be stable.
Consider a portfolio P (t) ≡ N i=1 w i S i (t), where w i is the fraction of wealth invested in stock i. The portfolio return is given by R = N i=1 w i G i . The risk in holding the portfolio P (t) can be quantified by the variance . In order to find an optimal portfolio, we minimize D 2 under the constraints that the portfolio return is some fixed value R and N i=1 w i = 1. We thereby obtain a family of optimal portfolios, which we represent by plotting R as a function of risk D 2 [Fig. 5].
To find the effect of randomness of the C ij on optimal portfolio selection, we partition the time period 1994-95 into two 1-year periods. Using the cross-correlation matrix C 94 for 1994, and G i for 1995 [11], we construct a family of optimal portfolios and plot R as a function of the predicted risk D 2 p for 1995 [ Fig. 5(a)]. For this family of portfolios, we also compute the risk D 2 r realized during 1995 using C 95 [ Fig. 5(a)]. We find that the predicted risk is significantly smaller than the realized risk: [D 2 r − D 2 p ]/D 2 p ≈ 170%. Since the meaningful information in C is contained in the deviating eigenvectors that define business sectors, we construct a 'filtered' correlation matrix C ′ , by retaining only the deviating eigenvectors [12]. We repeat the above calculations for finding the optimal portfolio using C ′ instead of C. Figure 5(b) shows that the realized risk is now much closer to the predicted risk: [D 2 r − D 2 p ]/D 2 p ≈ 25%. Thus, the optimal portfolios constructed using C ′ are significantly more stable in time.
In summary, given only the change in price of a stock, and no additional information about the stock, we can partition the set of all 10 3 stocks studied into subsets whose identities correspond well to conventionallyidentified sectors of economic activity. The sector correlations are stable in time and can be used for the construction of optimal portfolios with a stable ratio of risk to return.
We thank J.-P. Bouchaud, P. Cizeau, E. Derman, X. Gabaix, J. Hill, M. Janjusevic, R. N. Mantegna, M. Potters, L. Viceira, J. Zou, and especially L. A. N. Amaral for stimulating discussions, and DFG grant RO1-1/2447 for financial support. Our results on the applications to portfolio selection were presented at the APS March 2000 meeting by one of us (BR) and independently by P. Cizeau.
[1] A recent review of research done by physicists is J. D.
[2] The 30-min data analyzed are from the Trades and Quotes database published by the New York Stock Exchange. The daily data is from the Center for Research in Securities Prices (CRSP) database. In both cases, only those companies that survive the entire period are considered in our analysis.
[4] V. Plerou, et al., Phys. Rev. Lett. 83, 1471(1999. [5] The presence of a very large eigenvalue, by preservation of the trace Tr (C)= N , tends to cause the other large eigenvalues and eigenvectors to be influenced by randomness. Therefore, in order to remove the effect of the "market mode" on other large eigenvalues, we implement the linear regression Gi(t) = αi + βiG 1000 (t) + ǫi(t), where We then recompute the correlation matrix C using the residuals ǫi(t). Alternatively, removing the intra-daily patterns in absolute values of Gi(t) using the procedure of Y. Liu et al., Phys. Rev. E 60, 1390(1999 also yields similar results. [6] An interesting classification of stocks into sectors using the concept of ultrametric distances was obtained by R. N. Mantegna, Eur. Phys. J. B 11, 193 (1999). [12] To filter C from the effects of the random part, we first construct a diagonal matrix Λ ′ , with elements Λ ′ ii = {0, . . . , 0, λ988, . . . , λ1000}. We then transform Λ ′ to the basis of C, thus obtaining the 'filtered' cross-correlation matrix C ′ . In addition, we set the diagonal elements C ′ ii = 1, to preserve Tr(C) = Tr(C ′ ) = N .
[13] The curves specify ∆t = 30 min risk and return values.