A Workload-Specific Memory Capacity Configuration Approach for In-Memory Data Analytic Platforms

Abstract: We propose WSMC, a workload-specific memory capacity configuration approach for the Spark workloads, which guides users on the memory capacity configuration with the accurate prediction of the workload's memory requirement under various input data size and parameter settings.First, WSMC classifies the in-memory computing workloads into four categories according to the workloads' Data Expansion Ratio. Second, WSMC establishes a memory requirement prediction model with the consideration of the input data size, the shuffle data size, the parallelism of the workloads and the data block size. Finally, for each workload category, WSMC calculates the shuffle data size in the prediction model in a workload-specific way. For the ad-hoc workload, WSMC can profile its Data Expansion Ratio with small-sized input data and decide the category that the workload falls into. Users can then determine the accurate configuration in accordance with the corresponding memory requirement prediction.Through the comprehensive evaluations with SparkBench workloads, we found that, contrasting with the default configuration, configuration with the guide of WSMC can save over 40% memory capacity with the workload performance slight degradation (only 5%), and compared to the proper configuration found out manually, the configuration with the guide of WSMC leads to only 7% increase in the memory waste with the workload's performance slight improvement (about 1%)

\section{Introduction}
\IEEEPARstart
{I}{n-memory} computing frameworks, are widely adopted in big data processing. These frameworks keep the reused data among multiple tasks in the memory, which can reduce data processing time effectively, especially for iterative and real-time workloads.

As one of the representative in-memory frameworks, Spark[2] has the comprehensive ecosystem, which includes SQL query[3], machine learning, graph computing and streaming[4]. The comprehensive ecosystem makes Spark become the most widely used in-memory framework.
In Spark, memory is required for not only the reused data caching but also the data processing and data shuffling. Hence, the proper memory capacity configuration is a dominant factor of the performance guarantee of Spark workloads. Currently, Spark adopts the static way to configure the memory capacity for workloads based on user specifications. However, determing the proper configuration requires users to have deep knowledge about the target platform and their workloads. This is hard and cumbersome due to that users often employ a third-party packaged application and is not clear of its system-level characeristics. Hence, the non-expert users usually conservatively overestimate the memory consumption of their workloads and require in an excessive way. A recent analysis[6] on the resource utilization over 29 days trace period on Google’s big data center revealed that over 80\% of cluster memory are allocated to big data applications at most time, while around 50\% of them are technically unused. It is obvious that the user specification-based way will reduce the resource utilization significantly, especially under the multi-tenancy environment where the memory is shared and competed.

Guiding users based on the knowledge of platform mechanisms and workload characteristics is a natural and promising alternative on the memory configuration. However, this work is challenging due to two reasons. First, the efficient guiding requires the accurate prediction of the memory demand on each execution stage of the workload, which is not available yet. Second, such a {\textquoteleft}proper configuration{\textquoteright} is different across workloads due to the diverse memory consumption characteristics. However,
such consumption characteristics are not identified and classified among Spark workloads yet. To solve these problems, in this paper, for the Spark platform, we propose WSMC, a workloads-specific memory capacity configuration approach. WSMC classifies Spark workloads into four categories according to their memory consumption characteristic. For each category,the workload{\textquoteright}s memory requirement prediction model is established. On configuring the memory capacity of the ad-hoc workload, WSMC match the workload{\textquoteright}s memory consumption characteristic to one of the above four categories and determine the accurate configuration in accordance with the corresponding memory requirement prediction. Specifically, this paper makes following contributions:
\begin{enumerate}
\item
We explore the Data Expansion Ratio as the specific memory consumption characteristic to classify Spark workloads. The Data Expansion Ratio is referred as the maximum ratio of the shuffle data to the input data among the multiple processing stages of a Spark workload.For detail,according to the value range and the increasing rate of Data Expansion Ratio, Spark workloads can be classified into four categories, including Expanding.Rapid, Expanding.Medium, Medium and Shrinking. For an individual ad-hoc workload, its Data Expansion Ratio can be figured out by executing it with a small set of input data, and then, the corresponding category to be matched can be determined.
\item
We establish a workload-specific memory requirement prediction model for Spark workloads with the parameters of the input data size, the shuffle data size, the task parallelism and the data block size. Especially, we take the shuffle data size as the expansion of the input data size and determine the expansion factor for each workload category respectively. This model can predict the maximum amount of memory required among the multiple stages of Spark workload, and take it as the memory capacity configuration decision.
\item
We evaluate WSMC with SparkBench[7] workloads.For each workload, we execute it with three memory capacity configurations: the default configuration by Spark, the proper configuration found manually, and the configuration guided with WSMC. Compare to the default configuration, the configuration with the guide of WSMC can help to save over 40\% of memory capacity with the workload’s performance slight degradation (only 5\%). Compared to the proper configuration, the configuration with the guide of WSMC leads to only 7\% increase in the memory waste with the workload's performance slight improvement (about 1\%).
\end{enumerate}
\par
The rest of the paper is organized as follows. In Section 2, we discuss the background and our motivation. Section 3 introduces the methodology of workload-specific memory configuration. Section 4 is the evaluation. Section 5 shows the related work. Finally, we draw the conclusion in Section 6.
\begin{figure}[!t]
\centerline{\includegraphics[width=2in]{memory_model.png}}
\caption{Spark Memory Management Model[1]}
\label{fig_sim}
\end{figure}

\begin{figure*}[!t]
\centering
\subfloat[PageRank]{\includegraphics[width=1.7in]{mem_pagerank.png}%
\label{fig_first_case}}
\hfil
\subfloat[ShortestPath]{\includegraphics[width=1.7in]{memory_shortestpath.png}%
\label{fig_second_case}}
\hfil
\subfloat[Terasort]{\includegraphics[width=1.7in]{memory_terasort.png}%
\label{fig_first_case}}
\hfil
\subfloat[ConnectComponent]{\includegraphics[width=1.7in]{memory_cc.png}%
\label{fig_first_case}}
\caption{Memory capacity requirements under different input data size}
\label{fig_sim}
\end{figure*}

\section{Background and Motivation}
In this section, we first discuss the memory management mechanism of Spark. Then, we describe our motivation by empirical observations of Spark workloads.
\subsection{Spark Memory Management}
Data is managed as memory abstractions called resilient distributed datasets (RDDs)[8] in Spark platform. The RDD is a collection of objects partitions across a set of nodes, and each node retains several blocks. One RDD block can be replicated across nodes for high availability, and can be recomputed according to the dependency of RDD. If the data lost due to the machine failure, computation is conducted in form of RDD actions and transformations, which can be used to organized the dependency of dataset as a DAG of RDDs.

The Spark workload is deployed as a \emph{driver program} running on a master node of a resource cluster and several \emph{executors} running on worker nodes. Executor launched as a JAVA processes where all tasks are executed in. Fig. 1 shows the typical memory model of the latest version of Spark. Generally, the memory of Spark executor consists of three parts: \emph{Reserved Memory}, \emph{User Memory} and \emph{Spark Memory}. \emph{Reserved Memory} is used for storing some internal objects of Spark, which set with a static size. Aside from this part, Spark allocates 25\% of the remaining heap memory as user Memory, this part of memory is managed by user totally. The rest of memory is managed by Spark runtime environment, which determines whether the workload is executed successful or not. This part of memory can be divided into two parts: \emph{Storage Memory} is used for caching RDD data and \emph{Execution Memory} is used for the intermediate data sorting, merging and shuffling during the data computation. Among Spark Memory, the scale and boundary of Execution and Storage part can be adjusted dynamically during the workload execution. Due to the fixed set of Reserved Memory, the main challenge of Spark workloads' memory capacity configuration lies on the configuration of Spark Memory, where most data processed and produced by the spark task are stored and most memory contention occur. Once the Storage Memory capacity is decided, the User Memory capacity can be made out according to the size ratio of 25\%:75\%. According to the analysis of Spark memory management, we can define the memory capacity requirement of Spark executor($Mem_{cap}$) as the following calculation:

\begin{equation}
Mem_{cap}=(Mem_{exe}+Mem_{sto})+UM+RM
\end{equation}
Where the $Mem_{exe}$, $Mem_{sto}$, UM and RM represent the Execution Memory, Storage Memory, User Memory and the Reservered Memory respectively.

Current approach adopted by Spark is to configure memory capacity based on user's specification. User should configure the memory capacity of each Spark executor by the parameter {\textquotedblleft}spark.executor.memory{\textquotedblright}. As the statement in[6], workloads have a tradeoff curve for memory configuration. Small configuration will cause the performance problem, while excess memory will waste memory resource. We can infer that there is a {\textquotedblleft}proper configuration{\textquotedblright} for a specific workloads, however, making such a proper configuration needs deep knowledge about their workloads and the runtime environment of the framework. \textbf{The {\textquotedblleft}proper configuration{\textquotedblright} is the optimum point by trading off the performance and the resource utilization, which can be find out manually.} Furthermore, such a {\textquotedblleft}proper configuration{\textquotedblright} is different across the diverse workloads, this is a tough challenge on memory capacity configuration.

\begin{figure}[!t]
\centerline{\includegraphics[width=2in]{different_workloads.png}}
\caption{Memory capacity requirements among different workloads}
\label{fig_sim}
\end{figure}

\subsection{Motivation}
Our motivation is that for Spark workloads, the memory capacity requirement can be classified and evaluated. This motivation is based on two major observations of Spark workloads. First, the input data size of the specific workload has a strong relevance to its memory requirements. Second, different workloads have different memory capacity requirements and their requirements can be classified. We conduct the study under the Spark 2.1, which is the latest Spark version. For workloads, we choose SparkBench, which is a comprehensive benchmark suite for Spark. All of the experiments are done in a two node cluster. Each node has a 4-core 3.3GHz Intel Core i5-6600 processors and 4GB memory.

\begin{enumerate}
\item
\textbf{The memory capacity requirement can be evaluated, and the input data size of the workload can reflect its memory requirements.} In order to study the memory requirement of a specific workload, we measure the proper memory configuration manually. We choose four workloads to evaluate the proper memory configuration under five different input sizes.
Fig. 2 shows that the input data size of the workload has a strong relevance to its memory requirement. With the increase of the input data size,  the memory requirement grows accordingly. On the other hand, the similar growing trend is shared among the observed workloads. Hence, we can infer that the workload's memory requirements can be evaluated and quantified as the monotonic function of the input data size.

\item
\textbf{Diverse workloads have diverse memory capacity requirement and their requirements can be classified.} In order to measure the memory requirement of different workloads, we choose 9 workloads with the same input data size, 600MB, the result is shown in Fig. 3. We can see that the memory requirements among different workloads are different, for example, for the same input data size, SVD++ and PageRank require the largest memory capacity of around 2.5GB, while the KMeans and LogisticRegression only require about 512MB. Therefore, the memory requirement across different workloads are vary widly. On the other hand, when the input size is same, these workloads can fall into multiple categories according to the amount range of the memory requirement.
\end{enumerate}

\section{Methodology of WSMC}
\begin{figure}[!t]
\centerline{\includegraphics[width=3.5in]{methodology.png}}
\caption{Overview of WSMC}
\label{fig_sim}
\end{figure}
\begin{figure*}[!t]
\centerline{\includegraphics[width=6in]{shuffle.png}}
\caption{Data processing mechanism of Spark}
\label{fig_sim}
\end{figure*}
\subsection{Overview}

As shown in Fig. 4, the methodology of WSMC includes two phases, offline phase and online phase. In the offline phase, first, with the typical benchmark, Spark workloads are classified into multiple categories based on the diversity in their memory consumption patterns. The key factors impacting on the workload’s memory capacity requirement are also decided. Second, for each workload category, with the consideration of the key factors, the workload-specific memory requirement prediction model is established. Then in the online phase, when user submit an ad-hoc workload, the workload is profiled on its memory consumption pattern and matched into one of the above categories. The workload’s memory capacity configuration is finally calculated in accordance to the corresponding memory requirement prediction.

\subsection{Key Factors Selection}

Besides the input data size, all the key factors impacting on the Spark workload{\textquoteright}s memory requirement need to be decided as the basis of the memory capacity configuration. As shown in Fig. 5, the data processing mechanism of Spark platform can be described as follows. The Spark application is organized as the DAG-like data processing stages. Each stage can be mapped into a set of parallel tasks. For each application, Spark launches a set of JVM-based executors to execute its stages. All the stages reuse the allocated executors and execute in serial. Tasks of the first stage will load the input data into the Storage Memory of the executors before the data processing through the {\textquotedblleft}data loading{\textquotedblright} operation. Each task in the first stage need to load a data block to process. For the following stages, each task will read a partition of the result data of its parent stage through the {\textquotedblleft}shuffle read{\textquotedblright} operation and store in the Storage Memory. Each task will also cache its own result data into the Execution Memory and flush to the local disk through the {\textquotedblleft}shuffle write{\textquotedblright} operation. In this paper, for each following stage, we call the total data read in {\textquotedblleft}shuffle read{\textquotedblright} operation and written in the {\textquotedblleft}shuffle write{\textquotedblright} operation as the shuffle data.
\begin{equation}
\begin{split}
Data_{shuf_{i}}=Data_{sr_{i}}+Data_{sw_{i}}
\end{split}
\end{equation}
Where the $Data_{sr_{i}}$ and $Data_{sw_{i}}$ represent the data size of {\textquotedblleft}shuffle read{\textquotedblright} and {\textquotedblleft}shuffle write{\textquotedblright} operation on the $i$th stage respectively, and the shuffle data size is the maximum value of $Data_{shuf_{i}}$.
\begin{equation}
Data_{shuf} = \max\{Data_{shuf_{i}}\}{i \in \{1,2,3...n\}}
\end{equation}
Where $n$ is the total number of stages. Besides, the reused input data will be cached in the memory if needed.

As mentioned before, in Spark, the memory capacity configuration is referred as the configuration of the individual executor. Besides the input data size, there are four more factors, including the shuffle data size, the data block size, the total task number of the following stages, the concurrent task number of an executor(which called task parallelism that we mentioned about) can have the impact on an executor{\textquoteright}s memory requirement. This is for that the data block size determines the data loading size of each task in the first stage. In the following stages, the shuffle data size, the total task number determine the data  size of each task{\textquoteright}s {\textquotedblleft}shuffle read{\textquotedblright} and {\textquotedblleft}shuffle write{\textquotedblright} data, while the concurrent task number of an executor determin the total data size that need to be hold on a executor. It is obviously that all these five factors can cover the main memory access behaviors of the Spark workloads.

Except of the shuffle data size, all the above five factors, can be obtained before the workload execution. However, with the knowledge of the workload’s characteristic, the shuffle data size can be deduced with the input data size. It is obvious that our model is feasible to predict the memory requirement of the Spark executor with the consideration of these five factors.

\begin{figure}[!t]
\centerline{\includegraphics[width=2in]{intermediate_data.png}}
\caption{Shuffle data size of different workloads under same input data size}
\label{fig_sim}
\end{figure}
\subsection{Classifying Spark Workloads}
\textbf{Classification Criteria of the Workloads}
As described in Section II, with the same input data size, the amount range of memory requirement of typical Spark workloads can be classified into several categories. However, the specific memory consumption characteristic taken as the criteria of the workload classification need to be figured out. Such characteristic can not only determine the memory requirement evaluation of workload categories, but enable the ad-hoc workloads to match into those categories.

We decide the specific characteristic with the consideration of the above five factors that impact on the workload{\textquoteright}s memory requirement. As described above, the input data size mainly determine the memory requirement of the first execution stage, while the shuffle data size determine that of the following stages. As Fig.3 shows, with the same configuration of data block size and total task number for shuffle operations, typical Spark workloads have diverse memory requirements even with the same input data size(600MB). Hence, the variety of the shuffle data size may determine the workload classification. To verify it, we examine the shuffle data size of workloads demonstrated in Fig.3. As shown in Fig.6, the workloads can also be classified according to the size range of the shuffle data. For the PageRank and SVD++, the shuffle data size is larger than the input data size. The shuffle data size of SVD++ and PageRank are over than 600MB, especially the shuffle data size of SVD++ is over than 1200MB. However, for the LogisticRegression and KMeans, their shuffle data size is less than half of their input data size. For all the other workloads, the shuffle data size is as much as 0.5 to 1 time of the input data size. More interesting, the resulting workload categories are quite same as those with the memory requirement classification. We therefore determine that the ratio of the shuffle data size to the input data size can be the specific memory consumption characteristic to classify Spark workloads and define it as {\textquotedblleft}Data Expansion Ratio{\textquotedblright}. The Data Expansion Ratio of the Spark workload can be calculated as follows:
\begin{equation}
\alpha=\frac{Data_{shuf}}{Data_{input}}
\end{equation}
Where the $Data_{input}$ and $Data_{shuf}$ represent a specific input data size and the corresponding shuffle data size respectively.

\textbf{Methodology of Classifying Spark Workloads}
We describe the methodology of classifying Spark workloads with the metric of Data Expansion Ratio.

First, the Spark workload is executed with a set of input data in turn. The input data set can be expressed as $DS$, $DS$ = $\{ds_1, ds_2, ..., ds_n\}$, which is sorted with the data size in ascending order. For each input data $ds_i$ the Data Expansion Ratio $\alpha_{i}$ can be calculated using formula(4). For the sake of accuracy, the Data Expansion Ratio of the spark workload is expressed as the average of all the Data Expansion Ratio.

Second, as shown in Table I, the spark workloads can be classified into three categories according to the value of their Data Expansion Ratio.

\begin{enumerate}
\item
\textbf{Expanding}: Workloads that generate much larger volume of the shuffle data than that of their input data.
\item
\textbf{Shrinking}: Workloads that generate much smaller volume of the shuffle data than that of their input data, generally less than half of the input data.
\item
\textbf{Medium}: The volume of shuffle data is between the above two categories.
\end{enumerate}

\begin{table}
\caption{Classification of the workload based on Data Expansion Ratio}
\centering
\begin{tabular}{|l|l|}
\hline\noalign{\smallskip}
Category Name & Classification Condition \\
\noalign{\smallskip}
\hline
\noalign{\smallskip}
Expanding& $\alpha \geq  1$ \\
Shrinking& $\alpha \leq  0.5$ \\
Medium& $0.5 < \alpha < 1 $ \\
\hline
\end{tabular}
\end{table}

Finally, as the Expanding category can cover a  very   large   range of the Data Expansion Ratio, which is from 1 to infinity in theory, we choose the {\textquotedblleft}Increasing Rate of Data Expansion{\textquotedblright} to do the further division. The Increasing Rate of Data Expansion of the spark workload is examined based on the input data set $DS$ and can be defined as follows:

\begin{equation}
inc_{shuf_i}=\frac{Data_{shuf_{i+1}}-Data_{shuf_i}}{Data_{input_{i+1}}-Data_{input_i}}(i\in{1,2,...n-1})
\end{equation}
And we calculate the average of $inc_{shuf_i}$ as the value of $inc_{shuf}$.

As shown in Table II, the expanding category can be further divided into two categories according to the workload’s Increasing Rate of Data Expansion.

\begin{enumerate}
\item
\textbf{Expanding.Rapid}: Expanding workloads with the Increasing Rate of Data Expanion greater than 2.
\item
\textbf{Expanding.Medium}: Expanding workloads with the Increasing Rate of Data Expanion less than 2.
\end{enumerate}

\begin{table}
\caption{Classification of the workload based on Increasing Rate of Data Expanion}
\centering
\begin{tabular}{|l|l|}
\hline\noalign{\smallskip}
Category Name & Classification Condition \\
\noalign{\smallskip}
\hline
\noalign{\smallskip}
Expanding.Rapid & $inc_{shuf} \geq  2$ \\
Expanding.Medium & $ inc_{shuf} < 2 $ \\
\hline
\end{tabular}
\end{table}

\begin{table}
\caption{The values of Data Expansion Factor}
\centering
\begin{tabular}{|l|l|}
\hline\noalign{\smallskip}
Class Name & $factor_{shuf}$ \\
\noalign{\smallskip}
\hline
\noalign{\smallskip}
Expanding.Rapid & 4(3+1) \\
Expanding.Medium & 3(3+0) \\
Medium & 2 \\
Shrinking & 1 \\
\hline
\end{tabular}
\end{table}
Base on the analysis above, we can divide the Spark workload into four categories, including {\textquotedblleft}Expanding.Rapid{\textquotedblright}, {\textquotedblleft}Expanding.Medium{\textquotedblright}, {\textquotedblleft}Medium{\textquotedblright} and {\textquotedblleft}Shrinking{\textquotedblright}.

\subsection{Prediction Model of Memory Requirement of Spark Workloads}

Due to that the memory capacity configuration in Spark is applied to each individual executor, we predict the maximum amount of memory that an executor requires among multiple processing stages and take it as the required memory capacity of a Spark workload. We establish the prediction model with the consideration of the four relevant factors described above. All the factors, except the shuffle data size, can be obtained via the configuration parameters. As mentioned above, the shuffle data size can be expressed as the expansion of the input data size. We predict the shuffle data size in a workload-specific way as follows:
\begin{equation}
Data_{shuf}=factor_{shuf} \times Data_{input}
\end{equation}
Where $Data_{input}$ is the input data size, $factor_{shuf}$ is the expansion factor. We evaluated nine workloads of SparkBench, and get the experiment value of $factor_{shuf}$. The detail $factor_{shuf}$ setting of different class is shown as Table \uppercase\expandafter{\romannumeral3}.

On the first stage, we mainly focus on the {\textquotedblleft}data loading{\textquotedblright} operation. Hence, we define the maximum amount of memory required by an executor on the first processing stage as follows:
\begin{equation}
Mem_{s_f} = Size_{block}\times min\{Task_{ex}, \frac{Data_{input}}{Size_{block}}\}
\end{equation}
Where the $Size_{block}$ represent the block size, the $Task_{ex}$ represent the maximum concurrent task number of an executor, which can be set by the parameter {\textquotedblleft}spark.executor.cores{\textquotedblright}. The $\frac{Data_{input}}{Size_{block}}$ represent the total task number of first stage.

On the other stages we focus on the {\textquotedblleft}shuffle read{\textquotedblright} and {\textquotedblleft}shuffle write{\textquotedblright} operations, the maximum amount of memory required can be calculated as follows:

\begin{tiny}
\begin{equation}
\begin{split}
Mem_{s_o}=\frac{Data_{input}} {Num_{ex}} \times \beta +\frac{Data_{shuf}}{parallelism} \times min\{Task_{ex}, parallelism\}
\end{split}
\end{equation}
\end{tiny}
Where the $parallelism$ represent the total task number for shuffle operations, which can be set by the parameter {\textquotedblleft}spark.default.parallelism{\textquotedblright}. The $\beta$ represent the condition whether the input data need be cached in the executor, set 1 as need to be cached in the executor else set 0. The ${Num_{ex}}$ is the number of executor, which can be calculated by the input data size, the the concurrent task number of an executor and the block data size. The calculation is shown as follows:
\begin{equation}
Num_{ex} = \lceil{\frac{Data_{input}}{Task_{ex}\times Size_{block}}}\rceil
\end{equation}
Therefore, the memory capacity of an executor is the maximum of the all stages.
\begin{equation}
Mem_{spark}=max\{Mem_{s_f}, Mem_{s_o}\}
\end{equation}

According to the Spark memory management in Section 2. The calculation of Spark executor's memory capacity configuration of each executor is calculated as follows:
\begin{equation}
\begin{split}
Mem_{cap}=Mem_{spark}\times \frac{4}{3}+RM
\end{split}
\end{equation}
where the $RM$ is represented \emph{Reserved Memory}, $RM$ is a static value(300MB in default).

All the fomulas of our prediction model can be represented by the key factors that we listed above, which are retrievable. Hence, our prediction model is feasible.

\subsection{Online Workload Profiling and Classification Matching}
A Spark environment needs to run the ad-hoc workloads. We cannot collect the shuffle data until it has been executed. This makes us difficult to make a proper memory configuration. WSMC addressed this challange by worloads profiling. We running the ad-hoc workload in a set of small-sized input data. For example, for a specific workload $WL$, first, we execute $WL$ in 10MB, 20MB, 30MB, 40MB, 50MB respectively, in corresponding, the shuffle data size are 20MB, 40MB, 60MB, 80MB and 100MB. Second, we calulate the Data Expanding Ratio of $WL$, the $\alpha$ of $WL$ is 2, which belongs to the
{\textquotedblleft}Expanding{\textquotedblright} workloads. Hence, we calculate the Increasing Rate of Data Expansion, the $ratio_{shuf}$ of $WL$ is also equals 2. Therefore, $WL$ is a {\textquotedblleft}Expanding.Rapid{\textquotedblright} workload. According to the Table \uppercase\expandafter{\romannumeral3}, the $factor_{shuf}$ is 4. We assume that the $Size_{block}$ is 128MB the $Task_{ex}$ and $parallelism$ are both 1, the $\beta$ is set as 1, when the input data size is 1GB, the memory capasity of $WL$ is 1153MB.

\subsection{Disccusion}
Through the description above, we propose a workload-specific methodology to predict the memory capacity configuration of Spark workload, including the classification methodology and the prediction model of Spark workloads' memory capacity. However, there are some conditions that our study not take into account, our prediction model assume that the input data size is lower than the total memory capacity of the cluster, this is due to user adopt the in-memory framework for taking most use of memory to improve the performance of their workloads. When the input data size is over than the maximum memory capacity of cluster, frquent data transformation between disk and memory will cause significant performance decline. Thus, our work not take this situation into account, while we will consider such sitiation in our future work.

\begin{table*}
\newcommand{\tabincell}[2]{\begin{tabular}{@{}#1@{}}#2\end{tabular}}
\caption{Workloads' memory capacity configurations guided by WSMC}
\centering
\begin{tabular}{lllcl}
\hline\noalign{\smallskip}
No & Name & Input Size(MB) & Category & \tabincell{c} {Memory \\ Configuration(MB)}\\
\noalign{\smallskip}
\hline
\noalign{\smallskip}
1 & SVD++ & 100 200 300 & Expanding.Rapid & 860 1160 1460\\
2 & PageRank & 100 200 300 & Expanding.Rapid & 860 1160 1460\\
3 & TriangleCount & 100 200 300 & Expanding.Rapid & 860 1160 1460\\
4 & ShortestPath & 400 500 600 & Expanding.Medium & 1040 1200 1360 \\
5 & Terasort & \tabincell{l} {600 700 800} & Medium & \tabincell{l}{1030 1130 1230}\\
6 & ConnectedComponent & \tabincell{l} {600 700 800} & Medium & \tabincell{l}{1030 1130 1230}\\
7 & SVM & \tabincell{l} {600 700 800} & Medium & \tabincell{l}{1030 1130 1230}\\
8 & PCA & \tabincell{l} {600 700 800} & Medium & \tabincell{l}{1030 1130 1230}\\
9 & DecisionTree & \tabincell{l} {600 700 800} & Medium & \tabincell{l} {1030 1130 1230}\\
10 & KMeans &  \tabincell{l} {1200 1300 1400} & Shrinking & \tabincell{l} {1040 1090 1140}\\
11 & LogisticRegression & \tabincell{l} {1200 1300 1400} & Shrinking & \tabincell{l} {1040 1090 1140}\\
12 & Matrix Factorization & \tabincell{l} {1200 1300 1400} & Shrinking & \tabincell{l} {1040 1090 1140}\\
\hline
\end{tabular}
\end{table*}

\section{Performance Evaluation}
In this section, we conduct experiments to evaluate the efficiency of the WSMC approach. First, we select twelve workloads from SparkBench as the workloads of experiments. Second, we choose the {\textquotedblleft}proper{\textquotedblright} configuration and the default configuration as the comparisons.

\subsection{Experimental Methodology}
We select twelve workloads from SparkBench as the workloads of experiments. Table \uppercase\expandafter{\romannumeral4} is the workloads' memory capacity configurations which guided by WSMC, we use five small-sized input data size to profile the workloads, which is 50MB, 100MB, 150MB, 200MB and 250MB respectively. As shown on the Table \uppercase\expandafter{\romannumeral4}, we evaluate each workload with three different input data size, and the corresponding memory capacity configurations are shown on the last column. In our cluster, the $Task_{ex}$ and the $parallelism$ are both 4, the $\beta$ is 1, and the $Size_{block}$ is 128MB. We execute all the workloads under three classes of configurations. The first one is the configuration under the guide of WSMC, the second one is the default configuration of Spark platform which is static configured as 2GB, the last one is the proper configuration, we get the proper configuration by trading off the performance and memory capacity, we increase the memory capacity of executor with fixed step until the curve of the performance and the memory capacity.

In order to evaluate the efficiency of the WSMC approach, we choose two metrics which is the execution time of the workload and the memory usage of the system.

\subsection{Execution Time}

Fig. 7 demonstrates the normalized execution time of workloads under different memory configurations. in the Fig. 8, we normalized the execution time of workloads under different memory configurations, and set the execution time of default configuration as the normalized object.

From Fig. 7, we can find that the execution time with the configuration under the guide WSMC and the proper configuration are slightly larger than that with default configuration of Spark. Compare to the default configuration of Spark, the execution time increase 4.68\% and 4.83\% respectively. However, their gap is very small.

On the one hand, comparing WSMC with default configuration, for different categories of workloads, we can see that the execution time of  {\textquotedblleft}Expanding{\textquotedblright} workloads increase about 5.5\% on average, the execution time of {\textquotedblleft}Medium{\textquotedblright} workloads increase about 4.5\% on average, and the execution time of {\textquotedblleft}Shrinking{\textquotedblright} workloads increase 3.7\% on average. We can also find that the performance penalty of workloads which belong to the same category are different, due to their different intermediate data size. For example, SVD++, TriangleCount and PageRank are belong to the {\textquotedblleft} Expanding, Rapid{\textquotedblright} application, while the average $\alpha$ of SVD++ is over than 10, but that of PageRank is about 2. Due to our approach only takes the main part of memory consuming, more intermediate data is more likely cause the memory contension, which can bring extra performance penalty. However, we can find that the performance penalty are quite slight, and with the execution time increased 6.25\% by maximum.

On the other hand, comparing WSMC with the proper configuration, the execution time with the configuration under the guide of WSMC is very close. The performance gap is about 1\%.
\begin{figure}[!t]
\centerline{\includegraphics[width=2in]{memory_all.png}}
\caption{The normalized execution time of different memory configurations}
\label{fig_sim}
\end{figure}
\subsection{The Memory Usage}
\begin{figure}[!t]
\centerline{\includegraphics[width=2in]{performance.png}}
\caption{The normalized memory usage under different memory configurations}
\label{fig_sim}
\end{figure}

Fig.8 demonstrates the normalized memory usage of workloads under different memory configurations. in the Fig.8, we normalized the memory usage of workloads under different memory configurations, and set the memory usage of default configuration as the normalized object.

As shown on the Fig. 8, compared to the default configuration, the configuration with WSMC can reduce the memory usage of the system significantly, and the minimal value is 36.5\% and the maximum one is 58\%. Lacking of deep knowledge of the Spark mechanism, non-expert users always adopt the excessive for ensuring the performance of their workloads, and the configuration always from the official document of Spark. From the experiments, we can see that using WSMC can reduce the memory usage of the system significantly. Furthermore, compare to the default configuration of Spark, the {\textquotedblleft}Expanding{\textquotedblright} applications can saves over than 50\% of memory, the {\textquotedblleft}Medium{\textquotedblright} application can save 40\% of memory, and the {\textquotedblleft}Shrinking{\textquotedblright} applications save 36.5\% of memory. Though the classification of workloads and the observation of the trend of memory requirement under different input data size.

On the other hand, compare to the proper configuration, the configuration under the guide of WSMC uses about 7\% of memory more than proper configuration. Furthermore, the {\textquotedblleft}Expanding{\textquotedblright} use 14\% of memory more than the default memory configuration, and the the {\textquotedblleft}Medium{\textquotedblright} uses about 5\% memory more than the default memory configuration, and the {\textquotedblleft}Shrinking{\textquotedblright} use 3\% memory more than the default configuration. From the result, we can find that the configuration under the guide of WSMC is more closely to the proper memory configuration than the default memory configuration of Spark.

\subsection{Summary}
Through our experiment, we prove that the memory requirement of Spark workloads can be evaluated and classified. According to the classification, we can predicted the memory capacity requirement of ad-hoc Spark workloads accurately. The memory configuration under the guide of WSMC only about 7\% memory wasted than that of the proper configuration.

Besides, WSMC model can reduce the memory usage significantly with slight performance penalty. Compare to the default configuration of Spark, the configuration based on WSMC can reduce the memory usage by 40\%, however the augmentation of execution time is only about 5\%.

So, we can find that the WSMC can guide the memory capacity configurations of Spark efficiently.

\section{Related Work}
The memory management for in-memory data analytic platforms is a hot topic.
MEMRUNE[10] is one of the dynamic memory management system for in-memory platform, which can adjust the space of caching and execution space dynamically and provide policies of data eviction for caching data. Besides, after Spark version 1.5, Spark community proposed a project named {\textquotedblleft}Tunsten{\textquotedblright}, the motivation of this project is to take the most use of the resource. For memory management, this project allows the Spark platform to manage off-heap memory directly, which can reduce the GC opeation and improve the performance of Spark workloads. Besides, this project also provide a cache-aware structure to process and store data efficiency. However, these works focus on how to  managing the memory space more effeciently, rather than memory capacity planning, our work and those works are complementary.

In addtion to the memory management, some researches are focus on the configuration optimization in order to tunning the performance of workloads. Some works provide some suggestions of configuration optimization by a comprehensive performance evaluation of target platforms. For example, Pan[9] et al analyze the I/O characteristic on Hadoop platform, they choose four typical big data workloads from BigdataBench[13] and measure the impact on the I/O characteristic through changing the key parameters, finally give some suggestions for improving the performance. The limitation of these works is that they cannot give a proper configuration to a ad-hoc workload. Some works aware the limitation, propose some online approaches to optimize the configuration. For instance, AROMA[14] and MRONLINE[12] is such system for Hadoop platform. Both of these two systems will first collect some performance data of workload that have been executed and construct a knowledge base. When user submit an ad-hoc workload, they will run these workloads in a small-sized data size or cluster with default configuration, collect their performance data, match them with the offline knowledge base, and the system will provide a proper configuration for this ad-hoc workload automatically. Our work learn from these approaches and applying them into multi-stage in-memory data analysis platform.

Besides, there are lots of works are focus on the optimization of Spark platform. Such as programming models[4] and APIs, on the other hand, some new hardware resources are integrated into the Spark platform, such as GPUs and FPGAs, user can utilize these resources by using the extended APIs to optimize their workloads, the module of resource management will manage these resources. At the same time, Spark is adopted in more and more domains, such as DNA Analyzing and Wether Forcasting, these works makes Spark plays more and more important role on data analysing.

\section{Conclusion}
In this paper, We propose WSMC, a workload-specific memory capacity configuration approach for spark workloads. In order to classifing the workloads, we take the Data Expansion Ratio, which is defined as the maximum ratio of the shuffle data to the input data among the multiple processing stages of a Spark workload,as the basic metric for the workload classification. Furthermore, for each workload category, establish a workload-specific memory requirement prediction model for Spark workloads with the parameters of the input data size, the shuffle data size, the task parallelism and the data block size. Through the comprehensive evaluations with SparkBench workloads, we found that contrasting with the defaultconfigurations, the configuration with the guide of WSMC can save over 40\% memory resource with a slight workload performance degradation (only 5\%) and compare to the proper configuration, which is found out manually, the configuration with the guide of WSMC leads to only 7\% increase in the memory waste with the workload's performance slight improvement (about 1\%).

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\section{Acknowledgements}
The work was partially supported by National Key R\&D Program of China under grant number 2017YFC0803300, Beijing Natural Science Foundation under Grant 4172004, National Natural Science Foundation of China under grant number 91546111, Beijing Municipal Education Commission Science and Technology Program under grant number KZ201610005009 and KM201610005022.

Distributed, Parallel, and Cluster Computing (cs.DC)


Ultra-low threshold polariton condensation

Abstract: We demonstrate condensation of microcavity polaritons with a very sharp threshold occuring at two orders of magnitude lower pump intensity than previous demonstrations of condensation. The long cavity-lifetime and trapping and pumping geometries are crucial to the realization of this low threshold. Polariton condensation, or "polariton lasing" has long been proposed as a promising source of coherent light at lower threshold than traditional lasing, and these results suggest methods to bring this threshold even lower.

\section{Introduction}

Early observations of microcavity polariton condensation\cite{Kasprzak2006,Balili2007} were novel tabletop demonstrations of quantum many body physics.  In contrast to the cold atom systems to date, polariton studies require modest optics and lasers and operate at liquid helium temperatures.  Being a superposition of a quantum well (QW) exciton and a cavity photon, microcavity exciton-polaritons inherit interactions and a light mass, meaning they can thermalize and still exhibit quantum phenomena at elevated temperatures.  The strong coupling of the exciton and photon can be achieved by placing QWs within a properly tuned optical cavity, much like the structure of the regularly-used VCSEL.  For a thorough review of microcavity polaritons see \cite{Deng2010}.

Indeed, the accessibility and robustness of the solid-state microcavity polariton has made it a promising candidate for technological applications such as optical switching\cite{Menon2010,Gao2012,Anton2012,Steger2012,Ballarini2013} and coherent light emission\cite{Deng2003,Nelsen2009,Schneider2013,Bhattacharya2013}.  However, the early reports of polariton Bose-Einstein condensation (BEC) were criticized because the short polariton lifetimes do not allow the gas to fully thermalize--these systems represent quasicondensates rather than thermodynamically equilibrated BEC.  Improvements in the mirrors integrated into these structures increased the cavity lifetimes from $\sim$ 1 ps to 15 ps\cite{Wertz2010}, and later to 135 ps \cite{Nelsen2013, Steger2013, Steger2015}.  Here we show that this improved lifetime and the proper selection of pumping and trapping geometry can dramatically reduce the critical pumping density to achieve condensation.

Polariton lasing occurs in the strong coupling regime.  At high carrier densities, the exciton-photon coupling saturates and the polariton states revert to exciton and photon states.  In this regime, the cavity enters traditional, rate-driven lasing rather than thermodynamic-driven condensation.  For a system with a lower condensation threshold, there may be a wider range of output powers below the transition to standard lasing.  In early condensation results\cite{Kasprzak2006, Balili2007}, the condensate exhibited initial spectral narrowing upon condensation, but rapidly broadened with increasing density as the system approached saturation.  With a lower threshold, our polariton laser exhibits a minimal broadening over orders of magnitude of density change.

To put our condensation threshold into context, we compare our current results to previous polariton condensation demonstrations in Table \ref{tab:review}.  We consider similar systems to our GaAs microcavity polariton with experiments conducted at moderate cryogenic temperatures (4-20 K) and excited nonresonantly with a laser tuned to create free carriers on the order of 100 meV above the lower polariton state.  Typically, these experiments use either a continuous wave (CW) or quasi-CW excitation, so we compare condensation thresholds in terms of the instantaneous pump power and intensity at the condensate.

\begin{table*}[htbp]
\small
\centering
\caption{\bf Literature polariton lasing thresholds}
\begin{tabular}{cccccccc}
\hline
Source
& \begin{tabular}{c} Lifetime\\(ps) \end{tabular}
& \begin{tabular}{c} Duty\\cycle \end{tabular}
& \begin{tabular}{c}Instantaneous\\power (mW)\end{tabular}
& \begin{tabular}{c}Instantaneous\\intensity (W/cm$^2$)$^{(1)}$\end{tabular}
& Trap type
& \begin{tabular}{c}Condensate \\size ($\upmu$m)\end{tabular}
& \begin{tabular}{c}Pump \\diameter ($\upmu$m)\end{tabular} \\
\hline
\cite{Kasprzak2006} & 1 & 1\% & 16$^{(2)}$ & 1670$^{(2)}$ & defect (random) & 16 & 35 \\
\cite{Balili2007} & 4 & 2.40\% & 33 & 6800 & stress (harmonic) & 5 & 25 \\
\cite{Wertz2010} & 30 & 100\%$^{(3)}$ & 5.0 & 160000 & lithographic (wire) & - & 2 \\
\cite{Nelsen2013} & 270 & 100\% & 5.0 & 9900 & untrapped$^{(4)}$ & - & 8 \\
\cite{Nelsen2013} & 270 & 100\% & 30 & 60000 & self-trapped (1-d) & 10 & 8 \\
\cite{Liu2015} & 270 & 1\% & 100 & 200000 & \begin{tabular}{c}stress and \\optical (ring)\end{tabular} & 60 & 8 \\
This work & 270 & 7\% & 0.57 & 13 & stress (harmonic) & 5 & 75 \\
\end{tabular}
  \label{tab:review}
\raggedright
\small
\noindent \\ $^{(1)}$ Intensities approximated assuming uniform pump density.\\
$^{(2)}$ Source is not clear if this reported power density is average or instantaneous.  Here we assume the power is instantaneous. \\
$^{(3)}$  Source does not specify chopping, so we assume it is CW.\\
$^{(4)}$  Condensate coherence is low in the untrapped phase.
\end{table*}

Several results have been left out from this assessment because insufficient details were included.  For example \cite{Azzini2011} used a pulsed laser but did not indicate the repetition rate, making it impossible to extract the peak powers.  Ferrier et. al. \cite{Ferrier2011} explored interactions in terms of power relative to the threshold, but did not indicate the actual powers used.

The high-quality, long-lifetime sample used in this work is virtually identical to that studied in \cite{Nelsen2013,Steger2013,Steger2015,Liu2015}.  The stress-based trapping geometry was of the same method as \cite{Balili2007}.  We thinned the substrate to $\sim$ 50 $\upmu$m, to achieve a sharp ($\sim$ 50 $\upmu$m diameter) and deep ($\sim$ 2 meV) stress trap.  Starting at a large photonic detuning (-20 meV), we applied strain, redshifting the exciton until the center of the trap was nominally at resonant detuning.  While a very tight pump focus can be used to create a local barrier\cite{Liu2015}, in this work we defocused the pump spot to be slightly larger than the stress trap (pump diameter $\sim$ 75 $\upmu$m).  The laser was injected at an angle of 20$^{\circ}$.

The sample was cooled in a coldfinger-type Montana Instruments cryostat held at nominally 6 K.  Optical pumping and thermal conduction through the stressor assembly likely mean that the lattice in the vicinity of the condensate is warmer than this.  To mitigate warming from the optical pumping, the pump laser was chopped at 2 kHz with a 7\% duty cycle using a mechanical chopper.

Another distinguishing aspect of this study was the use of a stabilized CW TiSaph laser (M$^2$ Solstis) which offers greater frequency and intensity stability of the pump.  Since microcavity polaritons have dielectric mirrors, optical pumping efficiency is highly spectrally dependent.  Instability of the pump laser therefore can lead to a fluctuating density, which can severely broaden the condensation phase threshold.

After the optimal excitation wavelength (nominally 730 nm) was determined for the stressed sample position, we conducted a power series study, tracking the luminescence spectrum from the lower polaritons accumulated in the trap. Figure \ref{fig:analysis} shows our main results.

\begin{figure}[htbp]
\centering
\fbox{\includegraphics[width=0.8\linewidth]{Analysis.pdf}}
\caption{Pump-density dependence of the trapped polariton gas. Frame (a) shows the integrated counts of the trapped polaritons.  See the text for the method of integrating this density.  The dashed lines are guides to the eye for linear proportionality, and the dashed-dotted vertical line indicates the BEC phase transition density. The fraction of trapped polaritons in the most-occupied state is presented in frame (b).  Frame (c): blue circles: the spectral FWHM of the trapped polaritons (i.e. an indicator of their coherence) is shown to dramatically drop above threshold; red asterisks: the blueshift of the condensate relative to the low density ground state stays well below the 6 meV Rabi coupling strength, indicating that the system is still in strong coupling.}
\label{fig:analysis}
\end{figure}

Figure \ref{fig:analysis} (a) shows the nonlinear jump in the total integrated luminescence of the trapped polaritons.  This sum of intensity counts was only conducted over the low-energy luminescence to exclude PL from outside the trap, which dominates the spectrum below the condensation density threshold.  The dashed lines are guides to the eye to highlight the linear relationships well below and well above the condensation threshold.  Frame (b) qualitatively indicates the fraction of particles in the most occupied state.  Below threshold, the most occupied state is not the ground state, due to bottleneck and thermalization effects.  Above threshold, this fraction underestimates the condensed fraction because it includes only the most intense spectral count rather than the sum over the coherent state.  However, together frames (a) and (b) clearly show that above threshold, polaritons are efficiently sucked into the condensate, with a relative reduction in the density of polaritons existing outside the trap or even in high energy states within the trap.

The blue circles in Fig.~\ref{fig:analysis} (c) show the dramatic spectral narrowing, measured by full width at half maximum (FWHM), of the polariton population within the trap.  In contrast to previous results (\cite{Kasprzak2006,Balili2007}), the linewidth of the condensed state stays pinned near our spectral resolution limit for more than an order of magnitude increase in pump power.  These earlier works showed an immediate increase in the spectral broadening of the condensate above threshold.

This can be clearly distinguished from traditional lasing because it occurs spectrally at the lower polariton energy, as shown by the red stars in Fig.~\ref{fig:analysis} (c).  If strong coupling were saturated, the lasing should occur at the cavity mode, which is 6 meV higher in energy than the lower polariton.  At most we observe $\sim$ 0.35 meV blueshift of the condensate energy over this extended pump power series study.

Following the work of Tassone and Yamamoto \cite{Tassone1999} we can estimate an upper limit of the density of excitons in the vicinity of the condensate.  The mean-field shift due to the interaction of excitons in QWs similar to ours is estimated to be $\sim 6$ $\upmu$eV/$\upmu$m$^2$\cite{Sun2015}.  Traditionally, the interaction strength of polaritons is assumed to scale according to their excitonic fraction, meaning that polaritons at resonance should interact with bare excitons with half this strength.  If we assume that the interactions with the bare excitons dominates the mean-field shift of the polariton states, we can express the spectral shift of this state to be $\Delta E = g n$ where $\Delta E$ is the blueshift of the polariton state, $g$ is the polariton-exciton interaction strength, and $n$ is the density of excitons.  At condensation onset we observe a 50 $\upmu$eV blueshift of the polariton ground state energy, which yields an estimate of 17 excitons/$\upmu$m$^2$.

The low threshold onset of polariton condensation can be qualitatively understood in terms of the longer polariton lifetime and the geometry of trapping and pumping.  Longer lifetimes means that the steady state will have commensurately higher populations for the same pumping powers.  Longer lifetimes also allow for better thermalization of the polariton gas.  However, as shown in \cite{Nelsen2013}, without trapping of the polaritons, they will travel ballistically away from the pump region, effectively reducing their lifetime and density.  Indeed, most untrapped polariton condensates exhibit reduced coherence \cite{Nelsen2013} or occupation of several states \cite{Wertz2010, Ferrier2011}.  While tightly focused pump spots can be effective for generating barriers on the fly \cite{Wertz2010, Ferrier2011, Liu2015, Sun2015, Sun2016}, it seems that the scattering into the low energy polariton states may be inhibited in the barrier region.  Here we have pumped a broad region with minimal renormalization and allowed the polaritons to cool and accumulate in the trap.

Whether lasing is achieved by optical or electrical pumping, the relevant parameter is the density of carriers at which coherence onsets.  Polariton lasing has been demonstrated to produce coherent light below the traditional lasing threshold, and here we have pushed the condensation onset to even lower absolute intensity by optimizing the polariton lifetime and trapping and pumping geometry.

The work performed at NREL is supported by the U.S. Department of Energy Office of Science, Basic Energy Sciences under DE-AC36-08GO28308; the work at Pitt was supported by the Army Research Office under Award No. W911NF-15-1-0466; and the sample growth at Princeton was funded by the Gordon and Betty Moore Foundation through the EPiQS initiative Grant GBMF4420, and by the National Science Foundation MRSEC Grant DMR-1420541.

We acknowledge D. Myers and J. Beaumariage for sample preparation and characterization.

Optics (physics.optics); Mesoscale and Nanoscale Physics (cond-mat.mes-hall); Quantum Gases (cond-mat.quant-gas)


$^{139}$La NMR investigation in underdoped La$_{1.93}$Sr$_{0.07}$CuO$_4$

Abstract: We report $^{139}$La and $^{63}$Cu nuclear magnetic and quadrupole resonance (NMR/NQR) studies in an underdoped La$_{1.93}$Sr$_{0.07}$CuO$_4$ single crystal, focusing on the $^{139}$La NMR in the normal state. We demonstrate that the local structural distortions in the low-temperature orthorhombic structure cause the tilting of the direction of the electric field gradient (EFG) at the nuclei from the c axis, resulting in two NMR central transition spectra at both the $^{139}$La and $^{63}$Cu nuclei in an external field. Taking into account the tilt angle of the EFG, the temperature dependence of the $^{139}$La spectra allowed us to determine the $^{139}$La Knight shift and the structural order parameter. The angle and temperature dependence of the $^{139}$La spectrum is in perfect agreement with the macroscopic average structure and proves a displacive transition. The $^{139}$La nuclear spin-lattice relaxation rates, $T_1^{-1}$, suggest that La$_{1.93}$Sr$_{0.07}$CuO$_4$ undergoes a gradual change to a temperature-independent paramagnetic regime in the high temperature region. Both the spectra and $T_1^{-1}$ of the $^{139}$La as a function of temperature reveal a sharp anomaly around $T_S=387(1)$ K, implying a first-order-like structural transition, and a dramatic change below ~70 K arising from collective glassy spin freezing.

\section{Introduction}

\lsco\ (LSCO) is a single-layered cuprate where the CuO$_2$ planes
are separated by a La$_{2-x}$Sr$_x$O$_2$ block layer. A mismatch
between the CuO$_2$ planes and the interlayer causes a structural
transition from a high temperature tetragonal (HTT) to a
low-temperature orthorhombic (LTO) phase,\cite{keimer92} where the
CuO$_6$ octahedra are tilted from the $c$ axis resulting in a
buckling pattern of the CuO$_2$ plane.
Further partial substitution of a rare earth element (Eu or Nd) at
the La site, or replacement of Sr by Ba induces another structural
transition in \lsco\ from the LTO to a low-temperature tetragonal
(LTT) phase. In the LTT structure, the local tilt of the CuO$_6$
octahedra persists but its direction is rotated by $45^\circ$ and
alternates by $90^\circ$ in neighboring planes yielding the
macroscopic tetragonal symmetry. Such a local structural
distortion as a function of doping has a large influence on the
magnetic and superconducting
phases.\cite{axe94,buchner94,bianconi96,suh99,klauss00} In
particular, the LTT phase seems to be a prerequisite for static
stripe order which may compete with superconductivity in
cuprates.\cite{tranquada95,grafe06,vojta09}

In view of the intimate coupling of the respective structure to
superconductivity, a detailed understanding of the \textit{local}
structure and its doping- and temperature-dependence is crucial to
understand the high-$T_c$ mechanism. Yet, there have been
unsettled issues regarding the local structure such as whether the
macroscopic average structure fully corresponds to the local one
involving a displacive structural
transition,\cite{friedrich96,braden01,simovic03a} or to a
superposition of coherent local LTT variants involving an
order-disorder
transition.\cite{billinge94,haskel96,haskel00,han02}

While nuclear magnetic resonance (NMR) as a local probe is
suitable to investigate the local structure, it is also extremely
sensitive to inhomogeneities in the sample which often complicate
the NMR spectra, leading to an erroneous analysis of the results.
For example, the magnetic phase separation
scenario\cite{julien99a} in LSCO with $x=0.06$ based on the
splitting of the $^{139}$La NMR spectrum turned out to be a
misinterpretation,\cite{julien01} although its true origin has not
been thoroughly understood.

In this paper, the local structure in a high quality \lscos\
single crystal is investigated via $^{139}$La NMR. The La site is
most strongly affected by the local distortions associated with
the structural phase transition,\cite{radaelli94,haskel96} making
the $^{139}$La a suitable local probe for the local structure.
Indeed, our $^{139}$La NMR results show that the local tilting
pattern of the CuO$_6$ octahedra in the LTO structure is
accompanied by a tilting of the EFG at the nuclei, resulting in
the splitting of the otherwise single resonance line. We confirmed
that the $^{63}$Cu spectrum splits into two in the LTO structure
as well, but with a much smaller tilt angle than the value for the
$^{139}$La. A strong temperature dependence was found in the
splitting of the $^{139}$La spectrum, which measures the tilt
angle of the CuO$_6$ octahedra which disappears abruptly above
$T_S$. Our NMR data thus provide an evidence that the macroscopic
average structure parameters are sufficient to account for the
local lattice distortions and their temperature dependence. In
addition, the $^{139}$La spin-lattice relaxation rates measured in
a wide temperature range are discussed.

\section{Sample preparation and Experimental details}

The crystal of \lscos\ ($T_c=14$ K) has been grown from 4N materials of
\ce{La2O3}, \ce{SrCO3}, and CuO using the TSFZ (Traveling Solvent Floating Zone)
technique. Using this technique, which is described in detail elsewhere,\cite{lambacher10}
large crystals of several centimeters length can be grown
under accurately controllable stable conditions (flux composition, temperature
and oxygen partial pressure), which in terms are the prerequisite for
homogeneous crystals.
The sample for the NMR measurements was cut out of an single crystalline rod
after the growth procedure along the crystallographic $a$, $b$, and $c$-axis using
the  Laue x-ray  method. The outer dimensions measured 1.7 mm,
2.5 mm, and 0.8 mm in $a$, $b$, and $c$-direction.

$^{139}$La ($I=7/2$) and $^{63}$Cu ($I=3/2$) NMR experiments were
performed on the single crystal of \lscos\ in the range of
temperature 4.2--420 K.  The sample was mounted on
a goniometer which allows an accurate alignment of the sample along the
external field. In
order to check the quadrupole frequency $\nu_Q$, the $^{139}$La
NQR spectrum at the $3\nu_Q$ ($\pm\frac{7}{2}\leftrightarrow \pm\frac{5}{2}$)
transition ($\sim18.9$ MHz) and the $^{63}$Cu NQR
spectrum at the $\nu_Q$ ($\pm\frac{3}{2}\leftrightarrow \pm\frac{1}{2}$)
transition ($\sim34$ MHz) were measured. The NMR/NQR spectra were
acquired by spin-echo signals as the frequency was swept through the resonance line using
Hahn echo sequence ($\pi/2-\tau-\pi$) with $\pi/2\sim2$ $\mu$s.

The nuclear spin-lattice relaxation rates \slr\ were measured at the central
transition of the $^{139}$La by the saturation recovery method and the relaxation
of the nuclear magnetization after a saturating pulse was fitted to
the following equation:
\begin{equation}
\begin{split}
\label{eq:T1}
1-\frac{M(t)}{M(\infty)}=
a&\left(\frac{1}{84}e^{-t/T_1}+\frac{3}{44}e^{-6t/T_1}\right.   \\
&+ \left.\frac{75}{364}e^{-15t/T_1}+\frac{1225}{1716}e^{-28t/T_1}\right),
\end{split}
\end{equation}
where $M$ is the nuclear magnetization and $a$ a fitting parameter that is
ideally one.

\section{Results and Discussion}

\subsection{$^{139}$La NMR spectra}

Figure \ref{fig:lasp1} shows the central transition NMR spectrum of
the $^{139}$La at 300 K and 10.7 T as a function of the angle
$\theta$ between the crystallographic $c$ axis and the external
field $H$. The $^{139}$La spectrum consists of two resonance lines
which both exhibit the same angle dependence with respect to $H$:
the peaks are centered at $\theta_m^\pm=\pm8^\circ$, respectively,
where the maximum resonance frequencies are observed. The two
angle-dependent sets of the $^{139}$La spectrum which are
symmetric around the $c$ axis imply the existence of two different
sites with the same occupation probability.

Such a strong angle dependence of the central
transition is expected from
second order quadrupole interaction of the nuclear quadrupole moment $Q$ that
is non-zero for the nuclear spin $I>\frac{1}{2}$, with
the EFG. For the
central transition ($-\frac{1}{2}\leftrightarrow \frac{1}{2}$), while the
first order quadrupole shift is zero, the second
order quadrupole shift is given by\cite{bennet}
\begin{equation}
\begin{split}
\label{eq:quad}
      \Delta\nu^\text{2nd} = \frac{\nu_Q^2}{32\gamma_n H}
      (1-&\cos^2\theta)\left[ 2I(I+1)-\frac{3}{2}\right.   \\
      &-\left. \left(18I(I+1)-\frac{27}{2}\right)\cos^2\theta \right],
\end{split}
\end{equation}
where $\gamma_n$ is the nuclear gyromagnetic ratio and $\nu_Q$ is the
quadrupole frequency given by $3e^2qQ/2I(2I-1)h$ where $eq$ is the EFG. Here,
without the loss of generality,
we ignored an anisotropy parameter $\eta$ which is non-zero in sites of
lower than axial symmetry.
From Eq.~(\ref{eq:quad}), one can see that when $\theta=0$, $\Delta\nu=0$ so
that the resonance frequency becomes one of the maxima and directly yields the Knight
shift since a quadrupole correction is not needed.

\begin{figure}
\centering
\includegraphics[width=\linewidth]{La_sp_angle.eps}
\caption{\label{fig:lasp1} Angle dependence of $^{139}$La NMR spectra
measured at room temperature and 10.7 T.  Solid
(dotted) lines  are theoretical calculations assuming $8^\circ$ ($-8^\circ$)
tilting of the principal axis of the EFG from the $c$ axis, using
$\mathcal{K}=-0.09$ \% and $\nu_Q=6.3$ MHz which were determined experimentally from this
study.  The anisotropy parameter $\eta=0.19(2)$ was obtained from the calculations.}
\end{figure}

It may be possible that the $^{139}$La is differentiated due to the presence of two EFG
strengths which would cause the different quadrupole shift at the nuclei. To check
the possibility, we measured the $^{139}$La NQR spectrum in zero field at the
$3\nu_Q$ transition.
Since a well-defined single NQR line as shown in Fig.~\ref{fig:lanqr} indicates
the unique value of the EFG in the system,
the two $^{139}$La peaks are attributed to the local tilting of
the direction of the EFG at the $^{139}$La
with respect to the $c$ axis,\cite{hammel93}  where $\theta_m$ corresponds to the
angle at which the direction of the EFG is parallel to
$\mathbf{H}$.

In order to confirm this quantitatively, we performed exact
diagonalization calculations of the nuclear Hamiltonian which can
be written as,

\begin{equation}
\label{eq:Ham}
\begin{split}
    \mathcal{H} =& -\gamma_n\hbar[1+\mathcal{K}(\theta)]\mathbf{I}\cdot\mathbf{H} \\
    & + \frac{\nu_Q}{6} \left[3I_z^2-I^2+\eta \left(I_x^2-I_y^2\right)\right],
\end{split}
\end{equation}
where $\mathcal{K}$ is the Knight
shift which is usually angle dependent and $I$ is the nuclear spin operator.

\begin{figure}
\centering
\includegraphics[width=\linewidth]{NQR_La.eps}
\caption{\label{fig:lanqr} $^{139}$La NQR spectrum measured at the $3\nu_Q$
transition at 40 K giving rise to $\nu_Q=6.3$ MHz.
The NQR spectrum from Julien et al.\cite{julien01}
measured in the sample of $x=0.1$ is compared. The narrower linewidth of our
NQR spectrum indicates less chemical inhomogeneity in the single crystal.  }
\end{figure}

In the calculations, we used $\nu_Q=6.3$ MHz from the
$3\nu_Q$ transition (Fig.~\ref{fig:lanqr}) and $\mathcal{K}=-0.09$ \%
(Fig.~\ref{fig:freq}). Here we neglect a possible angle dependence of
$\mathcal{K}$. Thus, $\eta$ is the only unknown
parameter in the Hamiltonian. By adjusting $\eta$, we accurately reproduced the resonance
frequencies of the spectra as a function of $\theta$, drawn as solid (dotted) lines in
Fig.~\ref{fig:lasp1}. From these calculations, the anisotropy parameter $\eta=0.19(2)$
was deduced. Since $\eta$ can absorb the effect
of the angle variation of $\mathcal{K}$, the actual value may slightly differ
from the one deduced here.
The excellent agreement of the theory with the experimental data
indicates a staggered pattern of the tilting of the EFG at the $^{139}$La
around the $c$ axis, due to the corresponding local structural distortions.

\begin{figure}
\centering
\includegraphics[width=\linewidth]{structure.eps}
\caption{\label{fig:structure} Structure of \lsco\ in the LTO phase at 10 K
for $x=0.075$, using the structure parameters reported in
Ref.~\onlinecite{radaelli94}, viewed
along [001] (a) and
along [100] (b). The unit cell is depicted as the dotted line.
The CuO$_6$ octahedra is tilted along
the [010]$_\text{LTO}$ (or [110]$_\text{HTT}$) direction by $4^\circ$.
In addition, La(Sr)-La(Sr) along the $c$ axis is tilted by $1.5^\circ$ in
opposite direction of the distortion of the \ce{CuO6} octahedra, being consistent
with a much larger tilted angle of the EFG at the $^{139}$La.
Note that the tilt direction of both the \ce{CuO6} octahedra and the La-La is maintained
along [010]$_\text{LTO}$,
whereas, along the diagonal direction i.e., [110]$_\text{LTO}$, it is rotated
alternately by 180$^\circ$, causing the so called \textit{buckling} of
the CuO$_2$ plane.  }
\end{figure}

As a matter of fact, this is well supported by the LTO structure
in Fig.~\ref{fig:structure}, which was drawn using structure
parameters for $x=0.075$ at 10 K given in
Ref.~\onlinecite{radaelli94}. Clearly, the main axis of the
CuO$_6$ octahedra is tilted by $4^\circ$ along the [010] direction
(or [110] in the HTT setting). In addition, the position of La(Sr)
with respect to the $c$ axis is slightly shifted in opposite
direction of the distortion of the CuO$_6$ octahedra, which may
cause a larger tilting of the EFG at the $^{139}$La than at the
$^{63}$Cu. Furthermore, the fact that the tilt direction of the
octahedra is reversed alternately along the [110]$_\text{LTO}$,
which induces the buckling of the CuO$_2$
planes,\cite{axe94,tranquada95} accounts for the two $^{139}$La
peaks whose intensities and positions as a function of $\theta$
are symmetric around the $c$ axis.

If the two $^{139}$La NMR lines are the result of the distorted
CuO$_6$ octahedra in the LTO phase, one should expect that there
are two Cu sites which are distinguishable in field as well.
However, the presence of the two central transitions of the
$^{63}$Cu has not been reported so far. It may be possible that
the tilt angle of the EFG at the $^{63}$Cu is too small to be
detected, especially with the relatively larger linewidth of the
$^{63}$Cu NMR spectrum than that of the $^{139}$La. In order to
check whether there exist two $^{63}$Cu NMR lines in the LTO
phase, the $^{63}$Cu spectrum has been carefully examined by
varying $\theta$ at 300 K. Indeed, as shown in
Fig.~\ref{fig:cusp}, the angle dependence of the $^{63}$Cu
spectrum unravels two central transitions which are centered at
$\theta_m^\pm = \pm 2^\circ$ respectively, being symmetric around
the $c$ axis. The solid (dotted) lines are from theoretical
calculations with Eq.~(\ref{eq:Ham}) using the values of
parameters, $\nu_Q=34$ MHz and $\mathcal{K}_c=1.18$ \%. Unlike the
case of the $^{139}$La, a non-zero $\eta$ is not needed to fit the
data. The negligible $\eta$ and the much smaller $\theta_m$ than
that of the $^{139}$La are also consistent with the LTO structure
depicted in Fig.~\ref{fig:structure}, where the La itself is
displaced. Therefore, we conclude that the two peaks at both the
$^{139}$La and the $^{63}$Cu are a natural consequence of the
HTT-LTO structural transition in \lsco.

\begin{figure}
\centering
\includegraphics[width=\linewidth]{Cu_sp_angle.eps}
\caption{\label{fig:cusp} Angle dependence of the $^{63}$Cu NMR spectra
measured at 300 K and 8.2 T.  The
solid (dotted) lines are theoretical calculations assuming $2^\circ$
($-2^\circ$) tilting of the direction of the EFG at the $^{63}$Cu from the $c$
axis, with parameters $\nu_Q=34$ MHz, $K=1.18$ \%, and $\eta=0$.}
\end{figure}

We are aware that the qualitatively similar argument was given by
Julien \textit{et al}.\cite{julien01} where they observed four
$^{139}$La peaks which were attributed to the tilting of the
direction of the EFG about the $c$ axis.  However, the four peaks
with various signal intensities and linewidths are difficult to
understand in detail and are not in accordance with only two
different La sites as expected for the LTO structure. We
conjecture that the sample used by Julien \textit{et al}.
consisted of two or more domains which have slightly different
orientations.

\subsection{$^{139}$La Knight shift and structural order parameter}

Next we discuss the temperature dependence of the $^{139}$La
spectra. In order to extract the Knight shift for $I>1/2$ in
non-axial symmetry, the second order quadrupole effect which
shifts the central transition, as given by Eq.~(\ref{eq:quad}),
should be taken into careful consideration, particularly for a
very small Knight shift which is the case for the $^{139}$La NMR
in \lsco. Taking advantage of the fact that the second order
quadrupole shift vanishes when $H$ is applied along the direction
of the EFG, we deliberately rotated the sample with respect to the
$c$ axis by $8^\circ$. Then, the second order quadrupole shift for
the peak p1 is zero and the intrinsic Knight shift as a function
of temperature could be extracted.

\begin{figure}
\centering
\includegraphics[width=\linewidth]{La_sp_10deg.eps}
\caption{\label{fig:lasp2} Temperature-dependence of the $^{139}$La NMR
spectra in an external field tilted by $\theta\sim8^\circ$ from the
crystallographic $c$
axis. In this case, p1 corresponds to the line in which the EFG at the
$^{139}$La is aligned along $H$, giving rise to the Knight shift for
$H\parallel c$ since there is no quadrupole shift. At 387 K, the two peaks
collapse to a single peak, indicating the LTO-HTT structural transition. At
low temperatures,
$^{139}$La spectrum is significantly broadened, implying the slowing down of the
spin fluctuations due to the spin-density wave (SDW) order. Each spectrum was
scaled with temperature to remove the Boltzmann contribution to the intensity.}
\end{figure}

With increasing temperature from 300 K, the separation between the
two peaks becomes smaller and the spectra collapse abruptly into
single spectrum at $T_S$ = 387(1) K, the structural transition
temperature $T_S$ from the HTT to the LTO
phase.\cite{keimer92,nakano94} This is another direct evidence
that the two $^{139}$La central peaks arise from the tilted
CuO$_6$ octahedra associated with the LTO structure. The
$T$-dependence of the resonance frequencies of the two lines is
shown in Fig.~\ref{fig:freq}(a). The single line above $T_S$ is
split into two at $T_S$, in which one line (p1) shifts up and the
other (p2) shifts down in frequency from the single line above
$T_S$. The line p1 gives rise to the Knight shift for $H\parallel
c$, $\mathcal{K}_c$, which is drawn in Fig.~\ref{fig:freq}(b).
Note that the Knight shift data above $T_S$ is obtained after
rotating back the sample along the $c$ axis, since the EFG should be
directed along the crystallographic $c$ axis in the HTT structure due to
the axial symmetry with respect to the $c$ axis.

Additional valuable information was obtained from
Fig.~\ref{fig:freq}(a). Just below $T_S$, the separation between
p1 and p2 rapidly increases but approaches a constant value at
low temperatures, that is, the separation between the two lines
$\Delta\nu$ as a function of temperature behaves as the structural
order parameter as shown in Fig.~\ref{fig:freq}(b). At $T_S$, the
transition is clearly discontinuous, indicating a first order
nature of the transition, in contrast to second-order character
reported in the diffraction measurement.\cite{axe94} On the other
hand, except for the discontinuous change at the transition, the
temperature dependence of $\Delta\nu$ shows the behavior expected
in a second order transition.

The narrow single resonance line in the HTT phase indicates that
within the resolution of the NMR experiment the local tilting of
the CuO$_6$ octahedra vanishes immediately above $T_S$ rendering
the local tetragonal symmetry, in contrast to the experimental
reports that the LTO-type local tilt remains even in the HTT
phase.\cite{haskel96,wakimoto06} Furthermore, the abrupt splitting
of the spectrum at $T_S$ confirms a displacive transition, not an
order-disorder one.

Since $\Delta\nu$ is a measure of the tilt angle of the EFG at the
nuclei, its rapid suppression with increasing temperature from 300
K toward $T_S$ may cause a finite second order quadrupole shift
affecting the $^{139}$La Knight shift which was obtained with the
tilt angle $8^\circ$ determined at 300 K. Then, the actual Knight
shift data just below $T_S$ may be slightly enhanced, in such a
way that data are smoothly connected to those in the HTT phase.
Likewise, $\Delta\nu$ itself should be affected with a small
error.

\begin{figure}
\centering
\includegraphics[width=\linewidth]{freq.eps}
\caption{\label{fig:freq} (a) Temperature-dependence of resonance frequencies of
two NMR peaks in Fig. \ref{fig:lasp2}. (b) Knight shift of the $^{139}$La is extracted
from the p1 line. The separation of the two peaks $\Delta\nu(T)$
behaves as a structural order parameter. $\Delta\nu$ reveals first order-like
discontinuous transition at
$T_S=387$ K, while it exhibits the second order-like temperature dependence except the region
of the transition. Solid line is a guide to eyes. Below 70 K, data deviate due to the
onset of the magnetic broadening associated with the incommensurate SDW order.}
\end{figure}

\subsection{$^{139}$La spin-lattice relaxation rate}

Figure \ref{fig:T1} shows the spin lattice relaxation rate \slr\ of
the $^{139}$La as a function of temperature in the range 4.2--420
K. With increasing temperature from room temperature, \slr\ is
rapidly enhanced. We found a discontinuous jump at $T_S=387$ K
which is followed immediately by a steep drop above $T_S$. The
sharp anomaly of \slr\ around $T_S$ is consistent with the drastic
change of the $^{139}$La spectrum at $T_S$ (Fig.~\ref{fig:lasp2})
and supports a first order structural transition. Upon further
increasing the temperature, \slr\ continues to increase up to 420
K which is a limitation of our equipment.

\begin{figure}
\centering
\includegraphics[width=\linewidth]{T1_La.eps}
\caption{\label{fig:T1} $^{139}$La \slr\ as a function of temperature.  A
very sharp anomaly at $T_S=387$ K was detected with the transition width less
than 10 K. Below 70 K, the critical slowing down due to the SDW order causes
extremely steep upturn of \slr.  Inset shows \slrt\ of the $^{139}$La versus temperature.
Note that \slrt\ increases more than four decades as the temperature increases
from $\sim100$ K down to 4.2 K.}
\end{figure}

Interestingly, the temperature dependence of $^{139}T_1^{-1}$ resembles that
observed in \lesco,\cite{suh99,curro00} where
$^{139}T_1^{-1}$ rapidly drops below
the LTT-LTO structural transition. The
HTT-LTO transition, however, is clearly manifested through an anomaly in the narrow
range of temperature
which is simply added on top of the rapid drop of \slr. Note that \slr\
already started to decrease at a much higher temperature than $T_S$.

Rather, we conjecture that the rapid upturn of \slr\ is associated
with a gradual change from a system with a local moment character to a
more itinerant system, as argued by Imai \textit{et al}.\cite{imai93} They
report that $^{63}T_1^{-1}$ of the $^{63}$Cu forms a broad maximum
near 200 K in similarly Sr-doped samples, approaching a
temperature independent value $^{63}T_{1\infty}^{-1}\sim2500$
s$^{-1}$ at sufficiently high
temperatures.\cite{imai93,fujiyama97} If the temperature and
doping independent $^{63}T_{1\infty}^{-1}$ at $T\sim
J_\text{ex}/2$, where $J_\text{ex}$ is the exchange coupling constant,
imply the exchange narrowed paramagnetic limit, one
may expect that $^{139}T_{1\infty}^{-1}$ should be dominated by
$J_\text{ex}$ and the hyperfine coupling
$A_\text{hf}$ according to the
relation,\cite{moriya56a,chakravarty90}
\begin{equation}
\label{moriya}
 \frac{1}{T_{1\infty}} \propto \frac{A_\text{hf}^2 \sqrt{S(S+1)}}{J_\text{ex}},
\end{equation}
where we neglect an anisotropy of $A_\text{hf}$ for simplicity. A
very rough estimation from the Knight shift values, which are
proportional to $A_\text{hf}$, obtained at room temperature gives
$^{139}T_{1\infty}^{-1} = {^{63}T}_{1\infty}^{-1}
(^{139}\mathcal{K}/^{63}\mathcal{K})^2\sim 10$ s$^{-1}$ which is
surprisingly close to the experimental values near 400 K. Then,
one may anticipate that with increasing temperature beyond 420 K
$^{139}T_1^{-1}$ increases further and decreases ultimately, or
gradually saturates, to a constant value that is close to $\sim
10$ s$^{-1}$, being temperature-independent.
This indicates that the $^{139}$La relaxation rates at high temperatures
near $J_\text{ex}/2$ are governed by the same local spin dynamics probed in the $^{63}$Cu
results.\cite{imai93} Therefore, the $S=\frac{1}{2}$
antiferromagnetic 2D Heisenberg model\cite{chakravarty90,gelfand93} may be indeed
appropriate to describe the high temperature spin dynamics in LSCO even at
$T\sim J_\text{ex}/2$.

Furthermore, we find that \slr\ decreases linearly with
temperature below 300 K down to $\sim150$ K. This is clearly shown
by \slr\ divided by temperature, \slrt\ (see the inset of
Fig.~\ref{fig:T1}). The Korringa-like behavior suggests that the
relaxation process of the $^{139}$La is dominated by the doped
hole carriers in this temperature range. Therefore, we ascribe the
rapid drop of \slr\ at high temperatures to an evolution from a
paramagnetic regime at $T\ge J_\text{ex}/2$ to a regime below 300
K where the scattering with doped holes dominates the spin
fluctuation properties, and thus the $^{139}$La \slr.

Below $\sim70$ K, \slr\ is enhanced in an extremely fast rate,
which is ascribed to the critical slowing down of spin
fluctuations associated with glassy spin freezing.  Such a rapid
upturn of \slr\ is compatible with the significant broadening of
the $^{139}$La NMR spectrum in the same temperature range as shown
in Fig.~\ref{fig:lasp2}. Down to 4.2 K, a maximum of \slr, which
gives a characteristic spin freezing temperature $T_g$, was not
identified. This is somewhat inconsistent with previous $^{139}$La
NMR results,\cite{julien99a,julien01} and suggests a reduced $T_g$
which is possibly related to less disorder in our sample.

\section{Conclusion}

Through $^{139}$La and $^{63}$Cu NMR, we proved that the local
tilting of the CuO$_6$ octahedra along [110]$_\text{HTT}$ occurs
in the LTO phase, whereas it is absent in the HTT phase in \lscos.
The tilt direction of the CuO$_6$ octahedra alternates along the
same direction resulting in the buckling of the CuO$_2$ plane. The
structural order parameter representing the tilt angle of the
CuO$_6$ octahedra was successfully obtained from the splitting of
the $^{139}$La resonance line as a function of temperature. The
structural transition at $T_S=387$ K is weakly first order and
displacive i.e., the order parameter is reduced rapidly with
increasing temperature approaching $T_S$, and abruptly vanishes in
the HTT phase. In other words, the average HTT as well as the LTO
phases are fully compatible with their local structures. Our data
also rule out the possibility of an inhomogeneous mixture of the
two phases in the whole temperature range measured in this study.

The $^{139}$La spin-lattice relaxation rates, \slr, exhibits a
sharp anomaly at $T_S$, which is consistent with the abrupt change
of the spectrum. The temperature dependence of \slr\ in the high
temperature region suggests that a
temperature-independent paramagnetic regime at high temperatures changes
gradually to a regime where the
scattering with doped holes dominates the $^{139}$La \slr. The
strong enhancement of \slr\ below $\sim70$ K is ascribed to the
critical slowing down of spin fluctuations associated with glassy
spin freezing.

\section*{Acknowledgement}

The authors wish to thank Markus H\"ucker for his helpful
discussions and communications. This work has been supported by
the DFG through FOR 538 (Grant No. BU887/4 and No. ER342/1-3) and
SPP1458 (Grant No. GR3330/2).